DOCUMENT RESUME 

ED 369 980 CE 066 462 



AUTHOR 
TITLE 

INSTITUTION 

PUB DATE 
NOTE 



PUB TYPE 



Nunan, Ted; Calvert, Jocelyn 

Quality and Standards in Distance Education. Report 
to the National Distance Education Conference. 
Deakin Univ., Victoria (Australia).; University of 
South Australia, Underdale. 
92 

152p.; Funded through the National Priority Reserve 
Fund and supported by the National Distance Education 
Conference Working Party on Quality and Standards in 
Distance Education. 
Reports - Descriptive (141) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC07 Plus Postage. 

Adult Education: ^Distance Education; Educational 
Administration; ^^Educat ional Assessment ; Educational 
Objectives; Educational Policy; '^Educational Quality; 
Foreign Countries; Higher Education 
Austral i a 



ABSTRACT 

A study to examine quality and standards in distance 
education identified the elements of quality, indicators that would 
signify their achievement, and processes that foster the achievement 
of quality. It focused on perceptions of quality of particular 
stakeholders. Documentation provided by institutions showed that they 
define and determine what they mean by quality of distance education 
against a range of objectives. Recoinmendations addressed issues of 
access and equity at national and institutional lev^* as factors in 
the quality of distance education. Concerns with appropriate policy, 
organization, and resource, distribution led to recommendations on the 
following matters: resources for academic staff development; 
development and use of appropriate technologies ; encouragement of 
excellence in distance teaching; and establishment of guidelines for 
disbursement of funds^ Recommendations designed to foster the quality 
of processes used in distance education involved consideration of 
course materials and their teaching and learning arrangements and of 
interactions among institutions, lecturers, and students. Indicators 
for monitoring and evaluating the quality of distance education were 
identified and organized under policy development, organization and 
resource management, and processes of distance education. (Appendixes 
include indicators of quality, views of teachers and institutional 
clients of distance education centers regarding quality and 
annotations of 57 selected works.) Contains 99 references. (YLB) 
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EXECUTIVE SUMMi?LRY AND RECOMMENDATIONS 

The focus on quality as a central theme in education in the 1990s is an outcome 
of the recent redirection and restructuring of higher education and is 
paralleled by developments in other countries. During this period, in which 
enrolment has increased and expanded to include a wider clientele, distance 
education and open learning have emerged as major vehicles for teaching and 
learning. 

The Project to Investigate Quality and Standards in Distance Education was 
developed under the auspices of the National Distance Education Conference 
through its Working Party on Quality and Standards. It was commissioned by 
the Commonwealth Government in 1991 and carried out under the guidance of 
a reference group. Its conception predated the release of significant and 
relevant Commonwealth publications, notably Performance indicators in higher 
education (Linke, 1991) and Higher educatioru quality and diversity in the 
1990's (Baldwin, 1991). Its completion comes at a time when the Higher 
Education Council is investigating the broader issue of quality in higher 
education. | 

The Project reviewed an extensive literature on quality, particularly in higher 
education, which deals with factors affecting quality, quality measurement and 
assurance, judging and evaluating quality, as well as the political and 
ideological issues surrounding its adoption as a focus of national and 
international attention. 

Quality of education is a construct which draws boundaries between what is in 
and what is out; what is acceptable and what is not; what is valued more and 
what is valued less. Meanings attached to this construct are embedded in the 
language of educational discourse, have a history, and are constantly being 
reshaped and reformulated. Investigating the quality of distance education is, 
therefore, a complex undertaking which is located in an inherited context of 
time, place and power. Further, the act of investigation itself reshapes parts 
of the construct. Also, any investigation, in its methodology and its choices of 
what to investigate, cannot be divorced from the circumstances that prompted 
the investigation or from possible policy initiatives that may arise from it. 
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Given this situation, the term quality defies any definition which will be 
universally accepted. When it is linked to performance, quality implies 
evaluation for comparative purposes; 'measures' of quality invoke norms and 
standards and judgements of quality are assisted throug*^ use of norm or 
criterion referenced indicators. Where measurement focuses on the student as 
a product of education, quality is seen as 'value added' by the process of 
education. When the emphasis is management of quality, attention focuses on 
strategies for achieving or improving quality. 

Quality also implies a combined judgement of 'how well' and 'how worthwhile'. 
In this context it must be asked who makes the judgements, on what basis, for 
whom and in whose interests. Judgements about quality are closely tied to the 
aims and interests of particular constituencies; they are owned by stakeholders. 

In addressing its terms of reference, the Project sought to identify three 
things: the elements of quality (what is valued in distance education); 
indicators that would signify that these aspects of quality have been achieved; 
and the processes that foster the achievement of quality or its improvement. 
Recognising different perspectives and levels of analysis arising from different 
constituencies, the investigation encompassed government policy statements; 
institutional mission statements, policies and services; the views of academic 
staff and those providing distance education services in institutions; and the 
expectations of institutional clients of Distance Education Centres, both within 
and outside the higher education system. Students and employers as 
stakeholders were outside the terms of reference and ascertaining their views 
must be the focus of other research. 

The Project has generated recommendations for fostering and improving the 
quality of distance education in a number of areas: 



Access and equity 

Worthwhile achievements may be outcomes that lie outside educational 
processes. Education at a distance, because it is closely linked to government 
goals for mass education and, in fact, is an instrument for achieving these 
goals, can be seen to serve the interests of the nation and the economy. While 
any national goals for higher education may apply to distance education, there 
are particular goals that are especially relevant to this mode. For example, 
one commonly acknowledged function of distance education is to widen access 
to higher education, thus contributing to the achievement of goals for social 
equity, lifelong learning, workplace education and links with industry. Thus, 
one set of recommendations addresses issues of access and equity at national 
CD?/- and institutional levels as factors in the quality of ^tance education. 
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Policy development, organisation and resource management 
Where value or worth lies within educational processes, distance education and 
face-to-face education (and vanants of the two) may be directed towards, and 
achieve, identical educational outcomes- While educational achievements are 
fully valued, regardless of mode of teaching and learning, it must also be 
recognised that the processes that achieve parity of educational outcomes vary 
according to mode. Thus, issues of quality become linked to appropriate 
policy, organisation and resource distribution. In distance education, this 
concern leads to recommendations on such matters as resources for academic 
staff development; the development and use of appropriate technologies; 
encouragement of excellence in distance teaching; and the establishment of 
guidelines for disbursement of funds relating to distance education activities. 

Processes of distance education 

Particular means, in influencing how well something is done, may be valued 
above others. Thus, there is debate about the relative value of processes 
employed in education and distance education. An example is support for 
greater use of communications technology and of the techniques of distance 
education because of their perceived educational value, particularly for adult 
learners. This involves both quality and efficiency considerations and relates 
to increasing diversity of teaching and learning methods. Recommendations 
designed to foster the quality of processes employed in distance education 
involve consideration of course materials and their teaching and learning 
arrangements and of the interactions among institutions, lecturers and students. 

Monitoring and evaluating 

Also central to national and institutional efforts to ensure the quality of 
distance education are monitoring and evaluation. Issues relate to 
administrative responsibility, institutional policy formation and efforts to 
monitor and report on the quality of distance education. Further, where 
institutional clients have particular expectations regarding the quality of a X 
purchased product or service, relevant information regarding such products or 
services should be available. Several recommendations address these matters. 



Further development 

The Project recognises that institutions have primary responsibility for 
designing effective quality assurance mechanisms* It is important to assess not 
only changes in quality but also the factors that contribute to these changes. 
Furthermore, it will be necessary to determine whether the introduction of 
policies and procedures may have unintended consequences that subvert the 
^ declared intentions. 
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The recommendations are addressed to the National Distance Education 
Conference, DEET, institutions engaged in distance education and schools and 
departments within these institutions. 

The Project has also developed a set of indicators which institutions may use in 
monitoring and evaluating their own performance in distance education against 
their particular missions and objectives. This is consistent with 
Commonwealth emphasis on institutional self-monitoring. 

It must be emphasised that a list of indicators compiled in such a way has 
particular shortcomings which signal the need for further work. First, the 
groups sampled did not cover the possible range of types of courses and mix of 
media with the possible result that what they emphasise is not universally 
applicable. Second, there are differences in emphasis among stakeholder 
groups which require further analysis. Third, relying on the views of current 
distance education practitioners gives credibility to current practice and voice 
to assumptions that may not be warranted over the long term or in all contexts 
in which distance education is practiced. Finally, as Linke (1991) has noted, 
the use of indicators may distort practices within institutions to conform to 
what is measured; where they are introduced, their effects should be monitored. 

While the Project has focused on distance education, its relevance to wider 
practices in higher education is claimed. It is recognised that there are 
conflicting views about characteristics which define and distinguish distance 
education and about the relation of distance education to other alternative and 
non traditional modes of education. For statistical purposes, DEET defines 
external enrolment as the case where 'all tmits of study for which the student 
is enrolled involve special arrangements whereby lesson materials, 
assignments^ etc., are delivered to the student, and any associated attendance 
is of an incidental, irregular, special or voluntary nature/ Other features of 
distance education include the use of non-contiguous communication and 
institutional service structures to support academics and students involved in 
distance teaching and learning. Clearly, other modes may exhibit some of 
these characteristics and the recommendations of the Project would apply. 

The focus and conclusions of the Project are compatible with current 
investigations of quality in higher education and of the use of performance 
indicators. Consequently, the recommendations and the work on indicators 
may be seen as complementing that work. 




Access and equity 

1 That within institutions involved in distance education, the person with 
designated responsibility for distance education (see Rec, 4) be 
formally involved in all planning and review processes relating to 
distance education, in particular in the setting of access and equity 
targets. 

2 That NDEC be actively involved in monitoring the extent to which 
distance education is employed in meeting national objectives for 
identified social groups. 

3 That DEET fund a national Project to: 

a evaluate academic program diversity at a national level, taking 
account of discipline, course, level of offer, admission 
requirements, quota, conditions of enrolment and delivery, and 
current levels of demand; 

b identify means of regularly monitoring academic program 
diversity both within and across institutions. 

Policy development, organisation and resource management 

4 That institutions engaged in distance education activities assign 
responsibility for distance education to a person at a senior level of 
administrative responsibility. 

5 That institutions establish specific guidelines for disbursement of funds 
relating to distance education activities, taking account of 
requirements for development (including research and course 
development) and delivery (including, besides academic costs, 
technology, academic and administrative support of students and 
residential schools). 

6 That in the allocation of teaching loads, departments or schools make 
adequate provision for the workload associated with the preparation 
and revision of course materials, including involvement in peer review. 
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7 That institutions ensure that students studying in all modes have 
equitable access to appropriate academic and administrative support 
(e.g. library, study skills, counselling and computing services). 

8 That institutions providing distance education involve staff with 
specialist expertise in teaching at a distance in their staff development 
programs. 

9 That in the planning and review of courses, staff with specialist 
expertise in distance education are represented on the plamiing or 
review committees that involve external or mixed mode enrolment. 

10 That institutions ensure that promotion criteria or codes of practice 
related to teaching recognise the quality of contributions to distance 
education including appropriate scholarship reflected in course 
materials. 

Processes of distance education 

1 1 That institutions consolidate and make publicly available their specific 
policies that deal with arrangements for off campus provision. 

12 That institutions establish p^^^^ies for the development, 
implementation and resourcing of .^e use of appropriate technologies 
in distance education. 

13 That institutiotiS review any policy statements on student rights and 
responsibilities to ensure their appropriateness to external students. 

14 That procedures for planning ar^ developing distance education, and 
monitoring and reviewing performance in distance education, in the 
area of course materials and their implementation address: 

format and presentational quality of learning materials; 
instructional quality including use of appropriate teaching and 
interactive processes; 
workload and assessment load; 
arrangements for student support. 
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Monitoring and evaluating distance education 

15 That institutions with a substantial involvement in distance education 
include in their annual reports major achievements or developments in 
the areas of: 

external course development and production; 
technologies used in distance teaching and learning; 
student support techniques; 
research and evaluation of distance education. 

16 That institutions involved in distance education identify and publish a 
list of input, process and output indicators that they intend to use in 
monitoring and reviewing performance in distance education against 
relevant institutional objectives. 

17 That all course materials used in distance education carry a statement 
of acknowledgement for input and support received from individuals 
and organisations involved in their development and production. 

Future developments 

18 That NDEC monitor institutional responses to the recommendations of 
this report by seeking specific information on particular areas covered 
in the recommendations. 
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Introduction 

The National Distance Education Conference (NDEC) was convened in 1989 at 
the time that eight Distance Education Centres (DECs) were designated as the 
principal providers of higher distance education in Australia- Its principal 
purpose was to coordinate, and foster cooperation in, addressing national goals 
for distance education. At its first meeting, NDEC established working parties 
to examine and report on areas of national interest. One of these was the 
Working Party on Quality and Standards. 

After discussions within the group and with NDEC, the Working Party reached 
the concliision that, while quality was much talked about in the context of 
distance education, there had been no serious attempt to define it or its 
elements. Furthermore, there was no consensus on the relative importance of 
various features that might be taken to contribute to the quality of distance 
education programs. As a result, a project was proposed to examine the issues 
and make recommendations on how quality and standards in distance education 
could be defined and assured. Further, it was defined as a developmental 
exercise in which those involved in different aspects of distance education 
would contribute to the development of a set of definitions and standards of 
practice. This plan was supported by NDEC and the Department of 
Employment, Education and Training agreed to fund the project. 

The call for expressions of interest from higher education institutions made 
reference to the fact that the assessment team that had reviewed applications 
' for designation as Distance Education Centres had considered ^the qiiality 
control mechanisms and systems est^iblished by the institution, the extent to 
which such mechanisms were formalised rather than at the discretion of the 
individual academic, and the capacity of the institution to rectify faults where 
these were identified.** It noted the need for: 

quality assurance in distance education; 

an agreed definition of the elements of quality; 

guidelines to enable DECs to assess their own operations. 

Addressing these concerns, the aim of the Project to Investigate Quality and 

standards in Distance Education was: 

to develop an inventory of standards that enables distance education 
provider institutions to define acceptable quality of service and to 
^ assess strengths and weaknesses in the provision of services as an aid to 

^^'^8 ^^^^ planning for quality improvement. 

"~" ■ 15 
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The anticipated outcome of the project was: 

to inform specialist providers of distance courses of the standards and 
services that are the goals in the national provision of distance 
education and to assist the overall evaluation of distance education in 
Australia. 

Terms of Reference 

The Terms of Reference for the project were as follows: 

1. Review the relevant literature on quality and standards in education, 
distance education and selected service industries and compile an 
annotated bibliography. 

2. Sxjrvey the eight DECs on their perceptions of what elements constitute 
quality in distance education and what procedures, policies and services 
they employ to ensure the quality of course materials and course 
delivery, seeking information on 

each service and what it provides; 

suggested criteria and levels of performance, encompassing both 
processes and products, which are thought to define acceptable 
quality. 

3. Develop an inventory of services available from the eight DECs in the 
areas of course development and course delivery. 

4. Seek ii^ut from potential client institutions on proposed standards for 
DEC development services as they relate to client satisfaction. 

5. Review current work on performance indicators and their application in 
higher educat with the aim of ensuring compatibility or 
comparability of any indicators developed through the investigation. 

6. Recommend an inventory of procedures and standards employing, where 
appropriate, performance indicators on 

the development procedures and the services provided by DECs 
that ensure the educational and production quality of distance 
education materials; 
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the teaching, studertt support and ancillary services that 
characterise satisfactory distance education programs. 

1. Prepare support materials, in the form of a guide or case study which 
illustrate the effective application of the indicators and standards in 
assessing the quality of distance education systems and services, place 
these standards in context, and provide advice about how to apply them 
and the difficulties that may be encountered. 

What is distance education? 

The view taken in this report is that there are no special characteristics of 
distance education which logically lead to conceptions of "teaching" or 
"learning" which would differ from the way we use those terms in relation to 
education generally. While it is recognised that there are conflicting views 
about characteristics which distinguish distance education and its relation to 
other alternative and non-traditional modes of education, there is nevertheless 
a practical way of identifying the enterprise of distance education. This 
practical identification follows the DEET definition applied for statistical 
purposes, namely, "Where all units of study for which the student is enrolled 
involve special arrangements whereby lesson materials, assignments, etc., are 
delivered to the student, and any associated attendance is of an incidental, 
irregular, special or voluntary nature". 

Other features of distance education include the use of non-contiguous 
communication and institutional service structures to support academics and 
students involved in distance teaching and learning. Different * modes of study* 
may use pedagogies commonly used in distance education and the 
recommendations of this Project may well apply to certain characteristics of 
these modes - distance education is not uniquely characterised by particular 
teaching and learning situations. There is, however, a need to look towards 
characteristics of institutional arrangements and the teaching and learning 
environment which attends to the needs of students who, for whatever reason, 
do not attend the institution at which they are studying. 

A later section of this chapter addresses the use of the terms open education 
and distance education in relation to this Project. 



Policy direction for distance education 

The project has its roots in broad policy directions outlined by the 
Commonwealth. The White Paper on higher education (Dawkins, 1988, 4.2) saw 
the Distance Education Centres as: 

playing a key role in achieving government objectives of growth and 
greater equity in higher education; 

enhancing the provision of external studies by reducing duplication, 
fostering cooperation between providing institutions and improving 
overall quality, availability and efficiency of external studies courses; 

providing a mechanism for reducing system-wide costs in the area of 
production of com se materials; 

providing a mechanism whereby all institutions might have access to 
accepted national external course materials and supporting 
technologies. 

In particular, the policy statement drew attention: 

the suitability of external studies for mature age students; 

external studies as the only means of access to higher education for 
rural and geographically isolated students, expressing a commitment to 
"further substantial expansion of external study opportunities for 
people from more isolated communities**; 

the existence of "parity of esteem" so that a qualification gained by 
external means is regarded equally with a qualification gained by 
on-campias study; 

the desirability of increasing the efficiency of delivery, in part through 
economies of scale resulting when particular course materials are used 
by comparatively large numbers of students; 

the growing trend towards the use of high quality instructional 
packages developed for external studies to improve the quality of 
teaching for internal students; 
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proposed developments in reporting where institutions [would] be asked 
to specify appropriate equity goals, strategies, priorities and measures 
of performance as an integral part of their educational profile 
statements. 

While these statements indicate a concern for quality, there was little 
indication of what was deemed to exemplify quality. The team that considered 
applications for designation as a DEC (Joint Department of Employment, 
Education and Training/Higher Education Council Assessment Team, 1989) 
referred to quality in only two contexts. First, they reported that in their 
visits to institutions they noted the quality of design and production of course 
materials but were unable to assess the quality of the academic content. 
Second, they concluded that the quality of student support warranted 
improvement although they did not state what would constitute such 
improvement. 



A later briefing paper on distance education in higher education (Department 
of Employment, Education and Training, 1990, 3) noted that DECs would: 



develop course packages for use by any institution teaching a 
particular discipline internally; 

collectively have the potential to assume responsibility for the course 
offerings of most non-DECs choosing to withdraw from external 
provision; 

be involved in "new course developments and quality improvement 
projects." 

It was in this climate that the terms of reference for the project were 
determined. A trial evaluation study on the use of performance indicators was 
yet to report and there was uncertainty in the higher education sector 
regarding the determination and use of quantitative and qualitative indicators 
and about what quality assurance mechanisms might be proposed by 
government. During the course of the project, however, three government 
publications set the parameters for the discussion of quality and its 
measurement in Australian higher education. In order to ensure the relevance 
of the project in thft discussion, it is necessary to review its terms of reference 
in the context of the framework they provide. 




Performance indicators in Mgfier education 

Performance indicators in higher edtication (Linke £jLal-» 1991), commissioned 
by the Commonwealth Government in 1989, marks fifteen years* investigation 
of means of improving accountability and efficiency in Australian higher 

19 
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education. The task of the Performance Indicators Research Group was "to 
develop and trial a broad range of quantitative indicators suitable for 
evaluating relative performance in higher education at both system and 
institutional levels and to report on their practicability, data requirements and 
appropriate conditions of use? (p9). 

The Project to Investigate Quality and Standards in Distance Education clearly 
has roots in this ongoing investigation and m ^y protitably take account of the 
premises, findings and recommendations of its more broadly based analysis. 

The report identifies three categories of indicetors that reflect "the different 
background characteristics and functions of higher education institutions" (pi 2): 

Instituaonal context indicators "provide a measure of various 
background or input characteristics which may influence institutional 
j)erformance." They "aid the interpretation of the indicators relating 
directly to institutional goals so that judgements about relative 
performance can take into account the specific context of each 
institution," (pi 7) 

Performance indicatars "measure the relative achievement of 
institutions and their constituent organisational units against their 
respective aims" [italics in the original]* (pi 7) 

Participation and social equity indicators concern "representation of 
disadvantaged groups and variety of educational provision" (pl2)« They 
"are in part a reflection of institutional context but may also provide a 
measure of performance against particular institutional goals." (pi 7) 

Several points made in the report are worthy of mention in the context of the 
present study: 

First, one criterion for vse as an indicator is "the existence of a valid and 
reliable data source", so that collation may be routine and accuracy "checked 
at the point of collection" (pi 8). Thus the report, in presenting each indicator, 
notes existing data sources and, where applicable, their shortcomings. This 
point is important for two reasons. Indicators must be judged on their 
practicality for collection over time and must readily be verifiable for 
purposes of comparison, both longitudinally and across the higher education 
system. Thir* limits the number and range of indicators that may be applied on 
a national scale. 
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Second, the report rejects the notion of "an ideal reference point or indicator 
value", pointing to "the essential diversity of aims and organisational practices 
which exist throughout the higher education system and which sustain its 
continuing development and adaptability to change" (ppl2-13)* Instead, it 
recommends that the range of indicators "allow different institutions to 
express their own areas of emphasis and to identify their strengths and 
weaknesses in performance on the basis of their particular profile" (pi 31). 

Third, while recognising that "educational indicators..xan only be interpreted 
in a meaningful way by reference to comparative values" (pi 2), the report 
cautions that meaningful comparisons require attention to context and 
changing pressures and must be interpreted in terms of underlying aims* 
Indeed, it recommends that "a system of implementation [make] appropriate 
provision for discussion of results and informed judgements on the possible 
consequences" (pi 31). 

Fourth, the report takes the view that quantitative indicators cannot "provide 
a comprehensive measure of educational quality" (131) and cautions against 
"mechanistic application of performance indicators as a substitute for, rather 
than an input to, qualitative judgement" (pi 34). In this view, indicators may be 
one source of information contributing to judgements of quality. There is a 
danger, however, that, once indicators have been selected, there will be a 
tendency for institutions "to direct their performance to the indicators 
themselves, regardless of what they reflect, rather than to the underlying 
issues of educational and research excellence, or indeed to any specific 
institutional goals." (pl31) 

The report uses organisational structures, educational content and research and 
development as its three basic dimensions of analysis. There i? no attention 
given to other ways of identifying subsystems — indeed, to use a subsystem 
classification which did not broadly reflect arrangements in higher education 
on a national level would have defeated the purposes of the evaluation study. 
Consequently, the use of external load as a dimension for analysis (and the 
consequent generation of indicators reflecting education at a distance) was 
outside the genesis and terms of reference of the Performance Indicators 
Research Group. This is not to say, however, that the report docs not have 
relevance in areas which apply equally to internal and external aggregations of 
students; however, there is a need for careful interpretation as the 
aggregations used in the study are not intended to distinguish issues of 
performance by mode of study. 
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For purposes of the present project, the report on performance indicators 
suggests that, where quantiative data are clearly defined and readily collected 
over time, such data may serve as useful input to inform a broader analysis of 
performance in distance education. In addition, quantitative information on 
aspects of distance education activity may provide useful background 
information for use, with other measures such as qualitative judgements, in 
interpreting overall institutional performance. The findings of the report also 
suggest that any quantitative indicators should be matched either to national 
goals for distance education at the system level or to particular profiles and 
strategies at the institutional leveL 

Finally, the particular indicators recommended in the report require analysis 
for their appropriateness to the distance education context. For example, to 
the extent that the Course Experience Questionnaire tested and put forward by 
the Performance Indicators Research Group reflects perceptions of classroom 
teaching, it will require adaptation for off campus courses. 

As another example, large numbers of off-campus students will influence the 
interpretation of typical statistics on measures such as student program and 
graduate employment (since most external students are in employment and 
study part time) thereby linking external load to these institutional context 
indicators. 



Hitter education: quality and diversity in the 1990*s 

The policy statement of the Minister for Higher Education and Employment 
Services (Baldwin, 1991) pinpoints quality as a priority, noting "the need for 
credible quality assurance arrangements for Australia's higher education 
system, and for arrangements to systematically reward excellence in teaching 
as well as research** (p2). The statement announces several initiatives **to 
promote institutional management of quality and to demonstrate that the 
system is meeting expectations** (p30). These include a reference to the Higher 
Education Council *'to investigate the characteristics of quality in higher 
education** (p30), grants from the Reserve Fund **to encourage and reward good 
teaching practices** and for projects **aimed at enhancing teaching quality** 
(p32), the creation of a National Centre for Teaching Excellence and a special 
fund to reward **good performance in the use of all available resources to 
attain the best quality'* (p33). 




The statement notes the Government's **particular interest in accountability 
for the use of public resources** (p29) and the fact that the **majority of OECD 
countries have some arrangements for quality assurance in higher education 
[frequently with] provision for quality assessments to be taken into account in 

an 
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funding decisions" (p34). The statement rejects such intervention for 
Australia, however, preferring instead a system of incentives coupled with 
institutional self monitoring and reporting in the context of institutional 
circumstances and goals. In formulating recommendation, the Project needed 
to take account of the philosophy and directions outlined in the statement. 
These concern, in particular, the roles and interests of institutions and 
Government, the locus of responsibility and the evidence that should be 
amassed 

The statement is clear in assigning responsibility co institutions 

to determine their mission, to define what they mean by quality and 
standards of performance against their own objectives, and to identify 
and provide the evidence necessary for them to gauge their success and 
satisfy their various stakeholders, (4,12) 

The emphasis is on institutional responsibility for developing their own 
standards and criteria for quality in the light of their missions and objectives. 
The implication for the Project is a set of suggestive guidelines and indicators 
that can be adapted to particular needs. 

The statement also stressed the importance of "management systems for 
resource control and performance monitoring" On the grounds that these "are 
essential to accountability which is the other side of the autonomy coin" 
(4.11), Applied to distance education, this suggests the appropriateness of 
attention to means of allocating resources and monitoring their use. 

With regard to performance indicators, it is stated that "The Government has 
no intention of prescribing performance indicators to be used by institutions" 
(4,14), Instead, it sees "indicators being of most use to institutions as part of a 
self assessment process rather than in direct application to funding at the 
national level" (4,13), This suggests that the Project aim at identifying 
indicators that will be most useful for institutional self-*monitoring. Besides 
quantitative indicators such as those devel(^)ed in the Linke et al, study, the 
statement also acknowledges the usefulness of "qualitative" indicators (4,15), 

The statement expresses Government's interest in "assisting institutions to 
establish and develop their management information systems" and in 
"supporting institutions' efforts to maintain and enhance quality of course 
design, teaching, assessment and staff development" (4,16), The Project may 
recommend, therefore, ways in which such assistance could be provided, 

po 
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Finally, the statement makes specific reference to the DECs in the context of 
technologial innovation: 

The Government believes that recent developments in communications 
and information technologies h^ye the potential to improve both the 
quality and efficiency of higher education provision. It believes that 
technological innovation, particularly the technologies developed in the 
operation of the DECs, has opened up opportunities for alternative 
delivery modes to be applied in higher education to enhance quality and 
increase diversity, (5.33) 

The Project thus should recommend procedures for relating innovative 
technologies to quality and efficiency. 

The quality of higher education 

The Higher Education CounciPs discussion paper. The quality of higher 
education, was published in 1991 to facilitate national debate on the 
characteristics of quality, strategies for fostering it, the factors affecting it, 
the relationship between quality and resources, and the means of monitoring 
changes over time. 
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The paper provides a range of ways to define quality and states the Council's 
position which can be summarised as follows: 

judgements about quality are appropriately made within universities, 
their faculties and their departments; 

there are a range of stakeholders in higher education — the 
institutions, students, employers, governments — who liave the right to 
e^qpect to be assured about the quality of higher education; 
it would be valuable to develop a common framework that could be 
used by institutions both in the ways they judge quality internally and 
the ways they assure external stakeholders of their quality. 

In developing a common framework, it is proposed that the central theme 
should be the experiences of graduates who have received a quality higher 
education. Judging quality in institutions should involve all aspects of their 
operations and the basis of judgement of these aspects should be their 
perceived function in contributing to graduate outcomes. This position 
represents a complex and multifaceted view of the factors that impinge on 
quality, the framework within which judgements are made, and the 
acceptability of the framework as appropriate for assuring external 
stakeholders. 2 4 
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The relationship between judgements about quality and stakeholders 
highlighted in the paper was reflected in the way the present Project 
attempted to focus on the perceptions of stakeholders identified in its terms of 
reference. The Project did not, however, address graduate outcomes. 

The Draft Code of Practice for University Teaching 

In December 1991, the Australian Vice-Chancellors' Committee released a 
draft code that was intended as a 'useful model against which universities can 
assess and adjust where appropriate their individual teaching practices'. 

The draft code contains statements which reflect the position established for 
the current Project. For example, ^the Project addressed such matters as 
institutional commitment through mission statements or aims, teaching 
excellence as a criterion for promotion, professional development programs to 
assist staff, review of new and existing courses, and workload issues. 

While the code of practice does not explicitly deal with teaching at a distance, 
it is clear that, by recognising the impact of institutional and departmental 
management in its statements of good practice, it can readily be applied to 
teaching and learning at a distance. 

The Project's approach to investigating quality 

The Project, which developed its approach before these reports were issued, 
chose to focus on perceptions of quality belonging to particular stakeholders. 
This decision war based on an analysis of quality (see Chapter 2) that highlights 
the central role of values in any appraisal of the quality of education. 
Embedded in stakeholder views are a complex mix of uses of the term which 
reflect various positions. For example, quality could be variously identified as 
the magnitude of change within a particular student or the acceptability of the 
level of student work in a course. That is, quality could be both open to 
comparative judgements (most likely when the learning outcomes of students 
were subject to wider review as in Masters level thesis requirements) and 
potentially beyond comparative judgement (where graduates of different 
disciplines bring different backgrounds to a graduate diploma course and the 
student outcomes, while meeting a minimum requirement, vary considerably). 
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The Project saw considerable value in investigating stakeholder views about 
quality because it could then ask stakeholders to participate in framing 
indicators. Following Franklin (1988) such an approach is process oriented, 
concerned as much with performance improvement as performance evaluation, 
and involves the active participation of those who may be evaluated. 

Such a participatory approach was seen as important to meeting the needs of 
institutions as they move towards self monitoring, evaluating and reporting 
their performance. Any indicators arising from this process would be: 

viewed as 'convenient proxy measures* with attention given to the 
various qualifications about use, appropriateness, validity, etc/, 
structured from viewpoints which positively reinforced the value of the 
role of the stakeholder in quality improvement. 

There are problems associated with this approach as there can be difficulties in 
combining ^bottom-up^ stakeholder views with 'top-down* management 
directives about quality. However, in avoiding resistance from those whose 
'view of reality* might be depreciated by judgements made by others about 
appropriate indicators and their use, it is hoped that indicators developed 
through the study can be used in a developmental way so that there is an 
integration of top-down and bottom-up priorities within the context set by 
institutional priorities. 

It is important to note that the investigation was limited by its terms of 
reference to investigating quality and standards employing the viewpoints of 
stakeholders such as government, educational institutions, client institutions 
and providers (professional and support staff working in distance education and 
academic staff.) The terms of reference did not involve student perceptions of 
quality of distance education. In the perceptional r*" the various stakeholders, 
however, it was clear that their views were structured around ways in which 
they might provide the best possible e3q>eriences and support for external 
students. 

Education at a distance, open learning and the Distance Education Centres 
The recent focus on the implications of increasing 'openness* in the teaching 
and learning milieu of higher education has been the subject of a discussion 
paper, Open learning, commissioned by NBEET and DEET (Johnson, 1990). 
Distance learning, considered as a category or mode, can be seen as a subset of 
open learning (Thorpe and Grugeon, 1987, 2). The implication of this is that it 
has been overtaken, outclassed and outflanked (Gawthome, 1990). 
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At the level of methodologies for teaching and learning, oj>en learning is an 
attempt to apply whatever technology and technique is thought appropriate to 
providing the learner with access and flexibility. These may involve a number 
of variables, including time, location, pace and choice of assessment in the 
course of study. The conventional dichotomy between face-to-face and 
distance mode thus is rendered irrelevant. Both the underlying philosophy of 
•openness' and the ways that technology can be used in educational settings 
have resulted in calls for recognition of a range of alternatives beyond internal 
or external mode. 

The use of a range of techniques and technologies is seen to constitute the 
•cutting edge^ of practice which draws support from the view that higher 
education institutions ought to utilise the best and most appropriate techniques 
available regardless of how such methods previously may have been 
characterised. Cross fertilisation between approaches should widen the range 
of instructional technique and delivery processes. Thus, face-to-face teaching 
will be changed by the employment of techniques of teaching at a distance; 
teaching at a distance may, through technology, address groups at a MocaV 
centre, thereby supplementing approaches which, through course materials, 
address individual learners. 

Thus these are strong arguments which: 

challenge the distinctiveness of education at a distance, and 
consequently, 

challenge the need for consideration of distance education as a special 
category for policy development; 

support the need to explore the benefits available to the institution and 
student if there were greater use of mixed or alternate mode delivery 
of courses. 

Clearly, the Project is charged with making recommendations about distance 
education, and inherent in this is a focus on what stakeholders identify as 
distance education. It will be important, however, that the Project 
acknowledge that some of its recommendations may well have wider 
applications linked to open learning. A central question taken up in Chapter 6 
is how to accoxmt for changing practices and possible future developments 
where the methodology and terms of reference of this Project have been 
necessarily focused. 
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Applying the Project's terms of reference 

Term of reference 1 involved a review of the relevant literature on quality and 
standards in education and distance education. The analysis of the literature is 
provided in Chapter 2 while a working bibliography and annotations of selected 
works are provided as appendices 4 and 5, 

Term of reference 2 involves a survey of the distance education centres on 
their perceptions of what elements constitute quality in distance education and 
what procedures, policies and services they employ to ensure the quality of 
course materials and course delivery. Taken with term of reference 3 which 
involves developing an inventory of services available from the distance 
education centres these two terms of reference have resulted in collecting 
data about policies (Chapter 3) and management for quality in distance 
education (Chapter 4). 

The perceptions of stakeholders have been collected together and expressed as 
indicators in Chapter 5, 

Term of reference 4 is to seek input from potential client institutions on 
proposed standards for distance education centre development services as they 
relate to client satisfaction. Appendix 3 explores perceptions from other higher 
education institutions and corporate clients (Education Engineering Australia, 
Telecom and the Australian Taxation Office). 

Term of reference 5 involves a review of current work on performance 
indicators and their application in higher education with the aim of ensuring 
compatibility or comparability of any indicators developed through the 
investigation. The analysis to meet this term of reference is provided in 
Chapters 1 and 2, 

Term of reference 6 involves recommending an inventory of procedures and 
standards employing, where appropriate, performance indicators on course 
development, teaching, student support and ancillary services. Chapter 5 
contains performance indicators while various recommendations are made in 
relation to their use. 

We have provided a range of materials as appendices so that the reader can 
consider any recommendations or work on indicators with reference to the data 
collected by the Project. 
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Chapter 2 

THE QUALITY OF DISTANCE EDUCATION - 
WHAT DOES IT MEAN AND HOW IS IT JUDGED? 

Introduction 

It is hardly an exaggeration to say that, just as funding issues dominated 
the past ten years, so quality issues will dominate the next decade. (The 
Higher, 1991) 

The focus on quality as the theme for the decade is an outcome of the 
redirection and restructuring of higher education. During this period of 
upheaval, distance education has emerged as a major vehicle for teaching and 
learning. This chapter examines the concept of quality and standards in 
distance education against the backdrop of higher education change and 
development and the broader debate about quality that has ensued, with 
particular reference to the current examination of quality in Australian 
distance education. 

The development of quality as an issue in higher educaticm 
The prominent place of quality and performance monitoring on the Australian 
educational agenda has parallels in other industrialised countries. It is. in f act» 
the current phase in the intervention by the state into the affairs of the higher 
education system. Institutional autonomy has been replaced by conditional 
autonomy as governments have sought to assert a management role over 
education systems in an attempt to direct them towards particular 
sociopolitical and economic ends. 

The widespread nature of this trend has been well documented. De Weert 
(1990), for example, analysed the reasons behind this intervention by the state 
and outlined its impact on systems in the countries of Western Europe. PoUitt 
(1990), in his review of the situation in British universities, recorded the 
appearance of externally imposed structures of quality control and 
performance monitoring in parallel with similar developments in continental 
Europe and North America. According to Wright (1989), this intervention by 
the state in Britain includes the measurement of performance, regulation of 
standards and efficiency improvement procedures. He views it as a means of 
monitoring any response of the system away from serving the material 
interests of those professions served by it towards new social and economic 
ends in tune with the political ideology of governments. He also contended 
that intervention has resulted because there has been a change in the public 
attitude towards the professions which goes beyond distrust of the ways in 
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which they use their power to a 'questioning of the nature of the esoteric and 
technical knowledge which has - traditionally - been the basis for claims to 
expert status'. (plSl) 

Interestingly, he is sceptical about any long term effects of intervention upon 
the system, believing that professionals will re-assert and reproduce- their 
power base by re-establishing 'the very vocabulary within which needs are 
apprehended and expressed and set the criteria for what is to count as 
education.' (pi 5 2) 

Pollitt (1990), < gain with reference to the British system, analysed the changes 
in higher education by employing Kerr's (1987) assertion that imiversities are 
experiencing a 'critical age'. While the first wave of criticism in the 60s 
centred upon 'greater opportunity', the theme of the second wave of the 80s 
was competence. The 'modernisation perspective' encapsulates four key 
assumptions or assertions about the present: 

...first, that this is a critical phase; second that the crisis concerns a 
failure on the part of the imiversities to measure up sufficiently rapidly 
to the 'needs' of contemporary society; third, that prominent among 
these imsatisfied needs are those for demographically broader access to 
higher education and for enhanced 'competence'; and fourth, that the 
more rapid acceptance of these needs by universities requires, and is 
calling forth, vigorous intervention by 'society' in what was until recently 
regarded as the universities' sphere of legitimate autonomy. (p61) 

The changes linked to this modernisation perspective were seen by Sizer (1988) 
to involve a change in the culture of institutions. He noted of universities in 
the United Kingdom that there are: 

...pressures on institutions, particularly universities, to change their 
'culture' from the free oligarchic and consensus model, supported by 
administrative styles of management, which assumes that strong 
professionals on lifelong tenure are allowed to regulate both resources 
and academic developments and judgements through collegial means...to 
an entrepreneurial and market economy model with executive styles of 
management. (pl53) 

The situation for German higher education was reviewed by Frackmann (1991) 
who commented: 



ERLC 



30 



-29- 



With higher education entering the 'mass production* era the public 
confidence in the institutions* *self management* capabilities with 
respect to quality seems to be somewhat injured. Without 
overemphasizing the analogy with industry, one might ask: What is it that 
forces industry to consumer orientation in the course of mass 
production? Undoubtedly, the search for an answer reveals the 
competitive environment, the existence of a market as being the driving 
force for consumer orientation and quality assurance in industry. No 
wonder that, while higher education has no longer an *elite* but a *mass* 
education function, public scrutiny is increasing as to whether higher 
education quality is controlled sufficiently by an environment functioning 
similarly as to what we witness in the economy. (p6) 

Kalkwijk (1991) reviewed the issue of quality assurance in the Netherlands and 
noted that the legislative framework there has resulted from an intention by 
government to intervene to a minimal extent in the affairs of the institution 
of higher education provided that they are able to demonstrate that education 
and research meet quality requirements. According to Vroeijenstijn (1991), the 
Dutch Government is interested in education quality assessment becaisse: 

1 The Government has the constitutional obligation to assure the quality of 
education. 

2 The Government is called to account to the Parliament for the spending 
of money. Can the Minister defend his decisions? 

3 The Government has its own aims with Higher Education, so mass-Higher 
Education is a political decision. (p2) 

Performance and quality in Australia 

Similar developments in Australia have been summarised by Linke et al. (1991) 
in a report on the xase of performance indicators: 

Performance appraisal in higher education has become a matter of 
increasing importance over the past 20 years or so throughout the 
developed world. The trend in Australia derives mainly from continuing 
pressures for expansion of higher education associated with general 
funding constraints. Partly as a result of these conflicting pressures, and 
partly because of the perceived slowness of change in higher education 
institutions, there has emerged a persistent and increasing call for 
improved efficiency and public accountability in all aspects of higher 
education, (pi) 
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This concern for university performance in Australia is punctuated by reports 
and activities spanning almost a decade. The Report of a study group on the 
measurement of quality and efficiency in Australian education (Linke et al., 
1984) and Quality measures iti universities (Bourke, 1986) examined the 
feasibility of assessing performance. More recently, the Senate Standing 
Committee on Employment, Education and Training posed quality of education 
as an issue of concern. Other developments such as the Second Tier Industrial 
Agreement, the Australian Vice-Chancellors' Academic Standards Program and 
the AVCC/ACDP Working Party on Performance Indicators have generated a 
climate of interest in issues surrounding 'competence', Hattie (1990), in an 
extensive review of performance indicators in education, linked these 
developments to the changes outlined in the Green Paper (Dawkins, 1987) and 
White Paper (Dawkins, 1988). Thus, the recent statements, reports and 
initiatives described in Chapter 1 were quite simply the culmination of a 
progressive process of development. 

It is significant that the exercise of examining quality was already underway in 
one part of the Australian higher education system through a Project funded by 
the Government. Distance education, which has received the special attention 
of governments around the world during the last twenty years, is an important 
element in the movement to mass education, lifelong learning and workplace 
training. It also has the dual qualities of making teaching explicit and open to 
review through its use of teaching and learning materials and of providing 
contrasts to conventional campus-bound education; both of these 
characteristics raise questions about parity of quality. 



For some, e national investigation of quality and standards in distance 
education was gratifying recognition that review of higher education will take 
account of the distinctive aspects of distance education and recognise that 
particular descriptors of quality framed from the general viewpoint of 
face-to-face provision may need refinement. For others, the fact that 
distance education was the first to receive attention supported suspicion that 
government was extending its intervention from general aspects of the system 
to specific subsystems, thereby signalling an intention to broaden its control. 
This fear may be assuaged, however, by the emphasis on self management in 
the Minister's policy paper (Baldwin, 1991). 
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The analysis of quality of education at a distance, according to the self 
management approach, should take account of the distinctive characteristics 
of this mode as part of a broader pattern of diversity and innovation and should 
provide guidelines for institutional management of quality as well as measures 
for judgement of performance* This analysis can only be useful, however, if it 
is linked to the general concept of quality in higher education and the 
framework set by government in relation to its intervention in higher education 
as a whole* 

Quality as approbation 

In discussion of higher education policy, the issue of quality often is raised in 
conjunction with concerns about expansion, structural change and funding 
constraints. In this context, the term quality signifies approval, commendation 
or satisfaction — in short, it provides a means of approbation. Used in this 
way as an all-embracing and general term, it does not have a precise meaning 
and therefore can serve various political and social ends. As Karmel et al. 
(1985) noted: 

for some it appears to serve as a synonym for excellence or efficiency, 
others use it as a metaphor for good educational practice and others 
again equate it with material provision. For many it is no more than a 
shorthand way of expressing value discontent with the present outcomes 
of education while covering up a lack of urgent policies and priorities for 
action.. .quality will always remain a subjective entity. (33) 



Declaring value discontent or approval can be an e)q>ression and exercise of 
power. Moodie (1988) commented that 'British imiversities, on the whole, seem 
to have believed that they exemplified quality and provided, in effect, an 
ostensive definition of it* (pll). Excellence and quality can therefore signal 
anti-egalitarian values; for example, the terms take on such meaning when 
they become part of the political rhetoric of those who believe *more means 
worse* in higher education. Quality of education features in debates when the 
culture of semi-elite higher education is changed by the needs of a semi-mass 
system. 

As an all embracing and general term, quality may also be used to approve or 
commend other general features of a system. Gale (1991), for example, linked 
originality and creativity with quality, stating that Equality is about variation, 
individuality, and it will only be maintained if we also maintain difference'. 
The conditions for achieving quality are thus the focus of her concern — where 
the term quality remains undefined there is value commendation about a range 
of factors which are thought to foster quality. Commenting on the German 
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higher education system, Gale concluded that in East German universities, 
'controls in the name of quality actually became controls in the name of 
intellectual death*. Quality, for Gale, is a factor of difference, autonomy and 
freedom. 

The notion that quality as a concept defies definition yet is recognisable in 
practice is commonly held by those who use the term to commend or approve. 
Quality (and excellence) describe judgements of those who possess implicit 
knowledge about the practice of the area under consideration — for example, 
quality research is a judgement about an activity which can be made only by 
those engaged in research. This view is behind positions such as that taken by 
the Academic Audit Unit of the Committee of Vice-Chancellors and Principals 
of the United Kingdom that Mt is for the universities to say what they mean by 
quality and standards, and to show how they are achieved' (Williams, P.R. 1991). 

Quality measurement and comparison 

When it is linked to performance, quality implies evaluation for comparative 
purposes. T. Williams (1988) noted that: 

...definitions of quality tend toward something like the degree of 
excellence, relative goodness, grade. The question to be asked then is 
'relative to what?' Two p>ossibilities are suggested. First, we can assess 
quality relative to some accepted standard...if we are unable to measure 
achievements against some accepted standard in order to measure their 
quality then there may be something learned from comparison with the 
achievements of others. (114, 115) 

Such statements quickly move to consideration of 'measures' of quality and, in 
so doing, invoke norms and standards through the lise of norm referenced or 
criterion referenced indicators. 

Indicators are seen to assist in making comparative judgements or assessing 
achievement of goals. M. Smith (1988) outlined the link between comparison 
and indicators: 

An educational indicator is generally expressed as a number. It is 
intended to tell us something useful about the nature and health of 
education, whether at the national, state, local, school or classroom level. 

An indicator must also have a point of reference to give meaning to the 
number. A percentage has reference points of zero and 100. 
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Comparisons over time offer dates as reference points; comparisons 
among institutions, states, or nations have built-in reference points, as 
well. With the addition of reference points, indicators become more 
meaningful, more precise, more useful. In addition to points of 
reference, indicators assume some common understandings - the 
concepts of illiteracy, drop-out rate, and SAT score, for instance - 
whether or not such assumptions are justified, (p4S8) 

Numerous writers have signalled the danger of trying to reduce quality to a set 
of numbers. In proposing a set of performance indicators for Australian higher 
education, Linke et al, (1991) cautioned: 

The basic function of performance indicators is to assist in determining 
how well a particular institution or department has achieved its 
respective goals. In this respect they form an important part - but only 
part - of a broader evaluation process, in which their proper role as an 
aid to expert judgement rests on establishing a clear understanding of 
what characteristics are to be evaluated and what the indicators can and 
cannot measure in relation to these characteristics. 

It is apparent from the analysis of individual indicators that we may 
specify certain quantifiable measures of efficiency and effectiveness of 
performance and that these may provide useful profiles of institutional 
activity. However in no area of academic performance is it possible to 
generate systematic data which would adequately serve as the sole 
source of information leading to an acceptable evaluation of quality, 
(xiii) 

This view is reflected in the Minister's policy statement: 

The Government will support the further development of quantitative 
performance indicators, while bemg acutely aware of their limitations. 
The Government sees the main application of such indicators being at the 
institutional level as just one of a number of possible tools for measuring 
performance. (Baldwin, 1991, 4) 

Quality as magnitude of change 

A more specific application of measurement that focuses on the student as the 
product of education defines quality in terms of value-added. This is 'a 
measure of the difference between students' achievement at the beginning a 
programme of study and their achievement at the end' (Ramsden, 1991, 130) 
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and purports to reflect the extent to which an individual is transformed by the 
educational process* By focusing on magnitude of change rather than 
acceptance standards or absolute performance of graduates, such an approach 
aims to assess quality of education independent of its public prestige. For this 
reason, the value-added approach is particularly attractive for institutions that 
admit students without normal entry qualifications. 

Linke et aL (1991, 130) concluded that the value-added concept could not be 
applied in the assessment of quality in Australian higher education, 'partly 
because of the lack of any imiversal standards by which to assess student 
achievement across the diverse r;inge of higher education programs, and partly 
because of legitimate differences in expected educational outcomes or goals 
which cannot be applied across the system as a whole*. Moodie (1988, 11) 
pointed to the danger of wrongly attributing performance changes to 
institutional inputs when he commented that in such an assessment 'nursery 
schools and playgroups for the under-fives probably provide the highest quality 
education available'. 

In contrast, S.L. Smith (1991, 126-127), considering quality control and 
performance indicators for Canadian imiversity education, saw some promise 
for attempts to measure value added, but only in the assessment of core skills 
that may be expected of all graduates, such as written self-expression. By way 
of illustration, he drew attention to the initiative of one university where all 
students are assessed on entry and must complete a writing program within two 
years, noting that the initial assessment could be repeated upon exit to provide 
a measure of value added. 

It is interesting that in their consultative paper Quality tuition a working party 
of the Open University (UK) have employed the concept of value added linked 
with quality in terms of learning outcome to define quality. They note that: 

Quality tuition within the OU can be defined in two ways. Firstly it can 
be defined as process. It is the process which enables each unique 
individual student (within the immense diversity of the OU student 
population) to 'grow' as appropriate through active engagement with 
course materials, with the help of tutors and counsellors ... 
Quality tuition can also be defined in terms of the learning outcome. 
Quality tuition will thus have been achieved if, against the judgements of 
the constituencies ... a state is achieved when every student finishing his 
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or her studies with the University has been enabled to develop to his or 
her full intellectual potential and to his or her full satisfaction within the 
chosen area of study* (p2) 

Quality management 

The value-added perspectives treat the graduate as a product of the system; a 
consumer orientation treats education as a service 'product* which the student 
'buysV Frackmann (1991) attributed concern for quality in Germany to a public 
tendency to apply market concepts to their expectations for higher education 
as the system expanded. In some countries, however, higher education 
operates very much in the competitive market. This is the case in the United 
States, and particularly in the area of continuing education, where competitive 
advantage is a dominant concern (Baden, 1987). As higher education in other 
countries widens its market and becomes more entrepreneurial, the consumer 
orientation becomes salient. 

The 'consumer' can also refer to Government which, through funding public 
institutions, 'purchases' educational services. Marsh (1991, 16) commented 
that: 

The notion of quality which pervades much of Government thinking is 
strongly linked to the notions of service and accoimtability derived from 
the market philosophy which has been increasingly applied to all public 
services. Associated with this philosophy is the concern that public 
services should demonstrate value for money and responsiveness to 
idealised customers, free to shop around for the best available service 
they can find. 

According to Marsh, the commercial quality culture: 

...emphasises quality of service as its goal and operates through quality 
systems which will deliver measurable indicators of quality to potential 
customers. Such systems are supposedly neutral and objective, seeking to 
meet the stated or implied needs of the customer rather than pursuing 
some abstract notion of quality... 

The relatively passive term quality assurance Oiterally meaning to ensure 
the happening of quality) is rapidly being augmented by the harder-edged 
notion of quality management (implying the taking control of quality by 
executive action) and the associated requirement for quality systems, 
^ institutional audit and performance indicators. 
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While particular approaches to managing quality have received little attention 
in the higher education literature, allusion has been made to 'total quality 
management', described by Marsh as the 'evangelical' version of the 
commercial quality culture. This approach: 

focuses on customers (internal and external) and quality (which can be 
defined in terms of meeting customers' needs) as means of improving 
organisational performance. It emphasises the continual improvement in 
the quality and cost effectiveness of all processes, goods and services, 
through committed leadership and the creative involvement of all staff 
at all levels employing relatively straightforward statistical analysis 
techniques. (Cutting, 1991, p367) 

With total quality management, every worker has a customer either within or 
outside the organisation. To manage for quality improvement therefore 
depends upon Identifying the complex chain of inputs and outputs, noting where 
the outputs of one area become the inputs to another. While total quality 
management approaches have been employed in private industry and 
government departments, their impact in education is recent and so far 
appears limited to training programs, particularly those involving partnerships 
with industry. 

Quality and values 

The important place of values has been stressed in the quality debate. Linke et 
al. (1991) made this clear in an analysis of the relationship of quality to other 
terms linked to performance — productivity, efficiency, effectiveness and 
evaluation: 

For evaluation purposiss it is essential to distinguish between productivity 
in higher education, which reflects the effectiveness and efficiency of 
institutional operations, and quality of institutional performance. We 
tend to speak of efficiency of performance as referring to the level of 
achievement of certain specified goals in relation to particular inputs or 
efforts. It has no connotation of quality or value. We may speak of 
effectiveness of performance as referring simply to the level of 
achievement of institutional goals. It has no connotation of quality or 
value and no reference to particular inputs or efforts. Quality refers to 
judgements of excellence of performance and to the value or worth of 
what has been achieved. However, this does not necessarily reflect the 
significance or importance of the performance in question - 'importance' 
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refers to estimates made of an activity's influence in the broad arena of 
human knowledge and experience, and not to how well something is 
judged to be done. 

A statement about quality, then, is viewed most appropriately as a way 
of conveying comment on how well something is done and how worthwhile 
the achievement. Such a statement is relative and not absolute in that 
its meaning is best determined in relation to what others think. It has to 
be recognised, however, that perceptions of quality may be partly social, 
partly cognitive and partly political, and may also reflect factors other 
than the excellence of the activity, (pi 28) 

The last sentence is particularly important in pinpointing the basis of 
sensitivity in discussions of quality. Judgements about quality are inextricably 
bound to values held by social groups. Poole (1988) noted that: 

...fundamental questions concerning the meaning of quality need to be 
asked: (1) quality in terms of what (student satisfaction; performance 
indicators; market forces; quality of instruction?) (2) quality for whom 
(young people as clients of the system; for administrators as part of 
public accountability; for institutions of higher education; for 
employers?) and (3) quality in terms of whose interests (young people, the 
economy, the nation, international corporations?). (pSl) 

In this analysis, therefore, any determination of quality is a complex ejqjression 
of sociopolitical choice. Partington (1990) also made this point when he 
demonstrated that conceptions of quality are linked to particular constructions 
of education (the instrumentalist, the radical-reconstructionist, the liberal and 
the child centred) and pointed to the ways in which the structure of discourse 
about quality differs significantly between such constructs. Beneath these 
discourses lie a wealth of sociopolitical values which readily bubble to the 
surface, both influencing and forming the basis of judgements about quality. 

The quality of education at a distance 

For the purposes of this report the investigators chose to follow the work of 
Linke et al. (1991) in noting that statements about quality convey comment on 
how well something is done and how worthwhile the achievement. A statement 
about the quality of distance education, therefore, should contain both of these 
elements. 

o 3D 
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Worthwhile achievements may be outcomes of value or worth that lie outside 
educational processes* Education at a distance, because it is closely linked to 
government goals for mass education and, in fact, an instrument for achieving 
these goals, can be seen to serve the interests of the nation and the economy* 
While any national goals for higher education may apply to distance education, 
there are particular goals that are especially relevant to this mode* For 
example, one commonly acknowledged function of distance education is to 
widen access to higher education, thus contributing to the achievement of 
national goals for social equity, lifelong learning, workplace education and 
links with industry that are thought to contribute to economic, cultural and 
social progress* In this context, distance education is clearly the means to 
achieve such ends, often with the inference that education at a distance is 
particularly effective, especially where it enables particular groups to 
participate in educational processes* 

Statements about the quality of distance education also may focus upon 
worthwhile achievements within the educative process itself* Value is 
attached to certain generic skills, a body of knowledge and professional or 
technical skills considered to be the appropriate attributes of graduates, and 
thence to the academic content of courses and the communication channels 
thought necessary to achieve these aims* The inevitable comparison of 
distance education with other forms of teaching and learning may conclude 
equal achievement and, hence, 'parity of esteem'. Alternatively, distance 
education may be perceived deficient in its ability to convey particular 
academic content or develop particular skills or in its provision of 
communication channels and facilities* In the latter case, the conclusion of 
lesser quality may be drawn from assessment of individual elements of the 
educational package and process rather than from assessment of graduate 
achievement in the end. 

Thus, the quality of distance education may be assessed in terms of the 
achievement of sociopolitical ends, the performance of graduates and the 
mode's success in providing appropriate content and communication. There are 
three implications of this analysis. 

First, when value or worth is outside the educational process, distance 
education may be valued in itself as an effective means of achieving 
sociopolitical ends. In this context we can identify the ways in which national 
policies accord value to certain outcomes* Consequently, matters of access 
and equity, for example, become linked to quality of distance education* 

ERLC 



Second, where value or worth lies within educational processes, distance 
education and face-to-face education (and variants of the two) may be 
directed towards and achieve identical educational outcomes. While 
educational achievements are fully valued, regardless of mode of teaching and 
learning, it must also be recognised that the totality of processes that achieve 
parity of educational outcomes vary according to mode/ This totality of 
arrangements described by policy, organisation and resource distribution is 
valued in the way that it achieves excellence in outcomes - different mixes of 
processes may be valued less and represent inferior quality* 

Third, particular means, in influencing how well something is done, may be 
valued above others. Here debate is about the intrinsic value in particular 
processes employed in education and distance education. An example is 
support for widening use of technology or the techniques of distance education 
because of their perceived educational value. This involves both quality and 
efficiency considerations and relates to increasing diversity of teaching and 
learning methods. In-so-far as distance education is identified by its reliance 
on technology, such issues become part of the debate about quality of distance 
education. 

Tbe relevance of current literature in the field of distance education to the 
quality debate 

The literature of distance education is yet to analyse quality of distance 
education in terms of the areas identified above. Assumptions about what 
contributes to quality, however, are implicit in descriptive and prescriptive 
writing. In some cases, terms and procedures are adopted from industry. Guri 
(1987), for example, wrote about quality control systems and Lewis (1989) 
about "confora:ing to specification"; for both the specifications and processes 
are seen as the key to quality. Other writers have \ised parity with 
face-to-face education as a benchmark (Farren and Guitpn, 1987; Tate, 1986). 

Although some writers (e.g. Scriven, 1991; Viljoen, Holt and Petzall, 1991) have 
stressed the improtance of students* views, emphasising that the goal of 
education is to satisfy their needs, more often the focus is on specifics such as 
the quality of course materials or the impact of a new technological 
development in allowing particular processes to occur thereby affecting the 
quality of distance education. For example, the Australian and South Pacific 
External Studies Association Biennial Forum held at Charles Sturt University in 
1991 which addressed the theme 'quality in distance education* saw the 
majority of papers on how aspects of practice 
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could be linked to quality improvement. Papers addressed areas such as 
technology including audioconferencing (Thompson, D. 1991), television, 
(Moodie, 1991), videoconferencing, (Harris and Hague, 1991) and Email and 
computer applications (Howse, 1991; Thompson L, 1991); course and 
instructional materials development; (Bottomley 1991; Warn, 1991; Cauchi, 
Clift and Harris, 1991; Kemp, Nouwens and Towers, 1991); management for 
quality (Prebble, 1991, Inglis, 1991); quality in teaching and learning from the 
viewpoint of specific disciplines (Elliot, 1991; Mcllroy, 1991; Leys, 1991; Smith, 
1991). 

This tendency to assume that the experts know best has been criticised by 
Barache (1988) who describes the approach as "bringing to the consumer that 
which is assumed good for him" (pi 3). Paul (1990, p33) has pointed out that 
"because so little is known about how people learn and because of the 
overwhelming influence of personal, as opposed to institutional, factors on a 
student's performance, the [choice of strategies and services to increase 
interaction with staff and students] is often a roulette game, an expensive 
hit-or-miss approach with marginal impact on completion rates and increased 
costs per student." 

While it is the case that the majority of the works in the literature of distance 
education address quality from a limited framework or with reference to a 
particular process or aspect, there have been some attempts to provide a more 
global analysis, particularly where writers have adapted a Value added* or 
total quality approach* However, such approaches have had difficulties in 
identifying how to apply such concepts within the area of distance education 
(and the same sort of arguments about difficulties in applying the concept can 
be made for the general area of education as well). 

There is a further problem associated with exploring quality in distance 
education which arises from the 'industrialised* nature of the area - distance 
education is a collaborative undertaking where division of labour creates roles 
and responsibilities which arc more overtly managed than in general 
educational activities. There is often 'compartmcntalisation* of expertise and 
roles based around demarcation of authority. Within academic systems 
authority for educational decision making is frequently challenged, and 
resolution often through complex systems representing power groi^. Those 
professionals engaged in teaching see their role as central to the quality of the 
enterprise and have encapsulated this authority in the term 'academic 
freedom' - intervention into issues related to teaching and learning within 
disciplines is always challenged especially v^'here intervention is by others 
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outside of the academic or discipline based sul>-culture. Consequently, 
investigating teaching and learning at a distance by other than those teaching 
at a distance can be seen as an infringement upon academic freedom and 
therefore an illegitimate intervention. Thus much of the literature of distance 
education deals with issues which surround teaching such as events prior to 
teaching (the writing of course materials), advising on the choice of media 
delivery systems, establishing institutional student support, the outcomes of 
teaching (persistence, withdrawal etc), the impact of technology and resource 
management, etc. 

Using the framework provided involves iooki-:g iture v^ hich addresses 

the identified ways in which the term quality can be applied within distance 
education. Quality of distance education in terms of the sociopolitical ends of 
access and equity has been explored in the Anwyl, Powles and Patrick study. 
Who uses, external studies? Who should? (1987). Arger (1987) has also taken up 
sociopolitical issues in relation to Third World use of distance education while 
Nunan (1988, 1991) has elements of political analysis within writings about 
distance education in Australia. 

The second area, that of valuing the ways in which the totality of 
arrangements contribute to achieving excellence, is not well represented 
within the literature of distance education. Few studies have attempted to 
draw together such areas and ground their analysis in the practices which 
define the operation of the system. One study, Qmlity tuition (1990) however, 
is illtjstrative of this approach in that it attempts to address quality (from the 
viewpoint of a two-sided definition) in the areas of course production and 
maintenance, academic support and siq)ports which underpin the delivery of 
quality tuition. Recommendations focus upon policy, organisation and resource 
distribution to achieve quality outcomes from the viewpoints of particular 
stakeholders. Quality tuition is, in fact, a form of institutional review which is 
necessarily concerned with distance teaching and learning as it involves 
arrangements made by the British Open University. 

The third area, that of ascribing quality to particular processes employed 
within distance education is particularly well represented. Processes such as 
interactivity can l3e influenced by technologies, techniques employed within 
teaching, devices employed in writing course materials and the like - such 
processes are appraised from both quality and efficiency considerations. The 
previous listing of works from the •quality in distance education* conference 
illustrates this way of writing about the quality of distance education. 
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Applying the term quality within the project 

The Project to investigate Quality and Standards in Distance Education has 
worked to explore its terms of reference through adopting a broad definition of 
quality - namely, a statement about quality involves views on how well 
particular educational activities are done and how worthwhile the 
achievement. In addition, the Project has worked from the position that: 



1 Judgements about the quality of distance education can be made from 
the different frameworks of stakeholders. 

2 All stakeholders have the right to expect to be assured about the quality 
of distance education. However, the Project's terms of reference 
constrain it to a limited range of stakeholders. 

3 The Project will attempt to draw together views about factors which 
impinge upon the quality of education and provide recommendations 
which address managers, producers and clients of distance education. In 
drawing together these views the focus will be on how institutions 
manage for quality. 
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Chapter 3 

INSTITUTIONAL POLICIES - 

THE ROLE OF THE FRAMEWORK FOR DISTANCE EDUCATION 
IN DEFINING AND ACHIEVING QUALITY 

Introduction 

Institutional statements about distance education concern how well particular 
educational activities are done and how worthwhile the achievement. 
Consequently, they address both the quality of education achieved through 
distance education and the quality of educational processes within distance 
teaching and learning. 

As we have already seen, education at a distance, because it is closely linked 
to government goals for mass education and, in fact, provides an instrument 
for achieving these goals, is imbued with sociopolitical values. Institutional 
policies, like national policies, accord value to certain outcomes which are 
linked to distance education. The worth of these intended achievements form 
part of any consideration of quality. 

The Project sought published policy statements from each of the universities 
designated as Distance Education Centres. The policies were analysed to 
identify the range of concerns and commitments to complex value issues 
surrounding the institution's involvement with distance education. 

It may be expected that the policy statements from institutions broadly reflect 
the social purposes attributed to distance education through government policy 
initiatives. For example, A fair chance for all (1990) details ways in which 
government intends to increase access to higher education^ raising equity 
issues associated with the aim of enabling 'Australians from all groups in 
society (to) have the opportunity to participate successfully in higher 

education* in such a way that 'the balance of the student population reflect 

more closely the composition of society as a whole'. A fair chance for all sees 
a role for distance education in achieving greater access to education for the 
particular targetted groups identified. Institutional policy statements about 
distance education reflect such concerns and interpret these directions in 
terms of worthwhile achievements, often expressed, in mission or goal 
statements of the institution. How well the institution applies itself to 
meeting such missions or goals is also a factor of the quality of education at a 
distance. 
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National policy statements about the socially desirable goals of educational 
systems connect distance education to the quality of education through 
common themes such as concern for access, efficiency of delivery, parity of 
esteem, the purposes of mixed mode provision, the role of technology in 
distance teaching and learning, and equity. Consequently, it would be expected 
that these themes or variants would be translated to each institutional setting 
and featiH-e as a part of an institution's policies in distance education. 

Institutional policies related to distance education 

The documentation provided by institutions revealed a number of common 
features. The following outline selects statements from the range of 
(iocumentation to illustrate the context within which distance education is seen 
to operate and the general purposes distance education serves in meeting 
institutional goals. 

Institutions define and determine what they mean by quality of distance 
education against the range of objectives represented tlirough the following: 

1 Views about the need for education at a distance 

Illustrating the ways that institutions saw the need for education at a 
distance, are statements which identify a range of reasons why potential 
students are unable to undertake studies. The institutions indicate a 
general commitment to attending to these situations by providing 
distance mode courses. 

Typical examples are: 

... as the most geographically isolated ...(the institution) 
determined early in its istence that it would do all within its 
power to ameliorate educational disadvantage wherever and 
whenever it could. As a consequence, (it) has developed a range of 
programs to offer to persons who are geographically disadvantaged 
as well as to those who for reasons of physical disability, work 
commitments, domestic circumstances or other reasons are unable 
to undertake on campus study. 

... (the institution's) commitment to distance education means that 
the same opportunities can be offered to students who cannot 
attend on campus lecture programs because of work, family 
commitments or location. 
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The institution has a mission 'to maximise access to a range of high 
quality courses regardless of the student's place of residence, work 
commitments or mobility. 

2 Distance education and educationally disadvantaged groups 

There are, of course, access (and equity) issues related to how well higher 
education serves, and is used by, groups within the community which have 
low participation rates. 

Typical examples of statements of how distance education is involved 
with meeting the needs of educationally disadvantaged groups are: 

The commitment of (the institution) to increasing access to higher 
education generally and particularly through its external studies 
operation is second to none in Australian higher education. This 
can be demonstrated in relation to Aboriginal people, women, 
students with TAPE qualifications, the visually handicapped and 
those who are geographically isolated. 

Distance education... has provided the key to higher education 
access for many remote, isolated or otherwise disadvantaged 
students. It provides access to a higher than normal proportion of 
students of low socio-economic status.' 

...the centre's admission policy is sufficiently flexible to 
accommodate applicants with a variety of academic and work 
experience backgrounds. Applications are also invited from mature 
age people. 

The institution made available special entry for mature age 
students in rural areas. 

In external studies (the institution) offered...courses which were 
relevant to specific groups of employees who could not attend on 
campus i.e. prison officers, police officers etc. Pre tertiary 
courses for Aboriginal students and courses In Aboriginal Education 
were also offered. 
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«*«(the institution) aims *to develop and implement admission policies 
that recognise educational and social disadvantage in general and the 
particular disadvantage experienced by most Aboriginal people** 

Artiadation as an access issue 

To foster access to particular courses some institutions indicate a 
commitment to articulation* For example, 

•••a general Articulation Agreement (has been made) with TAPE to 
ensure easy transition and cross crediting for students who sought 
entry to higher education through post secondary TAPE College* 

«..the institution aims *to improve educational effectiveness by 
implementing programs which enhance articulation and credit 
transfer'. 

Cross enrolment and cross crediting 

To provide an extended choice of subjects within the university system, 
policies indicate support for cross enrolment and cross crediting 
arrangements. A policy statement indicated 'cross institutional enrolments 
are freely permitted.' 

The support for cross enrolment and cross crediting, within external mode, 
is seen as a way of avoiding utmecessary duplication of effort in the 
production of courses and their materials. Institutions typically note: 

A strong commitment to avoid unnecessary duplication of course 
offering and to foster state wide and national co-ordination through 
network arrangements for cross enrolment and course provision in 
the external mode. 

...(the institution) was to participate in the joint development of a 
state wide system of external studies and student support services in 
collaboration ... with other institutions. 

Further, (the institution) supports the determination of the 
Commonwealth both to maintain the present dispersed system of 
external studies provision and to achieve a desirable degree of 
rationalisation by reducing duplication where this is wasteful, 
fostering co-operation among providers and improving the overall 
quality, availability and efficiency of external teaching within the 
higher education sector. 



Mixed mode and flexibility 

Within a particular institution the availability of distance mode studies, as 
well as face-to-face tuition in the same area, is seen as a benefit. The 
resulting flexibility in the way such courses can be delivered increases the 
possibility that various students will be able to participate and progress 
through the course. Institutions note: 

Flexibility in administrative arrangements is seen to increase the 
options of external students and may therefore increase access. 
Course options are greatly increased through enabling students to 
choose a mixed mode of study. 

Mixed mode enrolment provides opportunities for study in both the 
internal and external modes concurrently or consecutively and thus 
provides opportunities for study for non school leavers and 
disadvantaged groups. The importance of this mixed mode of 
enrolment in relation to achieving the Commonwealth's equity 
objectives of improved access and progress of disadvantaged groups 
is stressed. 

...(the institution) has made this mixed mode enrolment option 
available and in any one semester one third of all OTHD students are 
enrolled in at least one external mode courf? and by graduation over 
sixty percent have experienced mixed mode study. 

Flexible teaching arrangements are also seen to assist disadvantaged 
students. 

In order to increase access for disadvantaged groups the institution 
has deliberately adopted an approach to course provision that 
emphasises the use of flexible learning and delivery strategies. 

...weekend and vacation schools provide access to other students, 
academics and library resources. 

Parity of esteem 

Parity of esteem between modes generally follows from mixed mode 
participation. Institutions note: 

Course flexibility ... (range of courses, flexibility in completion) 
was inherent in our initial brief in the development of two equivalent 
study modes. 
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Prima facie, all programmes (except veterinary studies) at ordinary 
degree, honours and masters by course work level may be developed 
externally. 

••.the provision of full majors was inherent in our initial belief in the 
development of two equivalent study modes, and it facilitated ease 
of movement between modes to meet changing student 
circumstances* 

The ^parity of esteem* between niodes provides the academic conditions 
for viewing external studies as an alternative and equivalent mode which 
should be potentially available to all students. Institutions 
characteristically note that external studies should be: 

•..a viable alternative learning mode available to students by choice 
rather than a substitute education for the geographically 
disadvantaged; 

available equally to students in the metropolitan area and in coxmtry 
centres 

7 Acadendc services 

The importance of equitable provision of academic services is addressed 
primarily with reference to the role of the institutions' libraries in 
meeting special needs of external students: 

The library provides a service for two groups of distance education 
students - those living in the vicinity ... who are expected to borrow 
in person, and those living further afield who are entitled to have 
material posted to them. 

Discussion and recommendaticMss 

The provision of distance education is clearly linked with a concern for 
providing access to university courses. The provision of access can be seen in 
terms of: 

providing, for any student who meets admission requirements and is 
selected, the means to study a course where the student either cannot, or 
chooses not to, attend the institution. 
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making available greater opportunities for those groups within the 
community which have low participation rates. Here access is more 
closely linked with concerns for equity, as policies specify targets for 
participation involving particular disadvantaged groups. 

Increasing access does not necessarily improve equity. While institutions would 
like to believe that increasing access will result in greater representation of 
disadvantaged groups in higher education courses, this is not necessarily the 
outcome of increased access. For example, increasing access through mixed 
mode provision may simply make it more convenient for those who are already 
over-represented in higher education. Also, mixed mode studies can be a 
pragmatic solution to overcome timetabling/staffing/space concerns. 
However, there are possible learning advantages for some students in being 
able to choose between different learning styles provided by intemid, external 
or mixed-mode provision. Further, the cross-fertilisation of methodologies 
and innovations in different modes is seen to be of value for both institutions 
and students. 

It is clear that institutions designated as Distance Education Centres see, as 
part of their mission, the achievement of particular objectives associated with 
access and equity as a function of their ability to provide distance education. 
At this level, quality and standards of performance are strongly linked to 
strategic decisions about the profile of courses and the external load in 
courses. Consequently, the translation of the objectives in these areas into 
targets and the mechanisms to report and evaluate progress towards such 
targets are important areas and involve management and monitoring of the 
external load of the institution. 

Recommendation 1 

That with in institutions involved in dis tance education^ the person with 
designated responsibility for distance education (see Rec. 4) be forn:ally 
involved in all plannin g and review processe s relating to distance education. \^ 
particular in the setting of access and eauitv targets. 

This recommendation is to ensure that institutions take account of the role of 
distance education in setting access and equity targets and that in involving 
the designated person responsible for distance education there is attention 
given to setting achievable targets within the resources available to support 
distance education* 
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Recommendation I 

That NPEC be actively involved in monitoring the extent to which distance 
education is employed in meeting natio nal objectives for identified social 

This recommendation arises from the need for national monitoring to inform 
decision making. NDEC can provide a suitable forum to monitor and appraise 
efforts which utilise distance education in meeting national objectives. 

RecQipnieTv fation 3 

That DEE T fund a national Project to : 

a evaluate academic program d iversity using distance education at a 
national level, taking acc ount of discipline, course, level of offer. 
admission requirements, quota, conditions of enrolment and delivery, and 
current level s of demand . 

b identify means of repularlv monitoring academic propram diversity both 
within and acro ss institutions . 

This recommendation addresses the need to identify, at a national level, any 
barriers to access of existing courses. In addition, it identifies elements of a 
database which would enable the identification of gaps in national provision of 
distance education thereby assisting policy formation at a national level. The 
recommendation is directed towards mapping and improving access to courses 
and providing information which relates current access to levels of demand, 
thereby providing a national picture of the role of distance education and other 
modes in meeting national objectives for access and equity. 
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Chapter 4 

MANAGEMENT FOR QUALITY IN DISTANCE EDUCATION 
Introduction 

The analysis of policy statements from institutions indicated emphasis on 
institutional or national goals for distance education. Interestingly, 'second 
order* policy guidelines to achieve such institutional or national goals, 
especially in relation to the management of resources, were not provided by 
institutions. 

It cannot be concluded from this that such policies do not exist, either in 
explicit form in the minutes of decision-making bodies or in the terms of 
reference of committees and organisational imits of the institution. De facto 
policies may also be reflected in institutional practice as an application of the 
rhetoric of mission statements. While acknowledging this, the need for 
codified guidelines in relation to resource management was recognised as a 
factor impinging upon the quality of distance education by stakeholders not 
involved with management or resource allocation within institutions. While it 
is recognised that there are considerable difficulties in addressing such 
resource issues it is felt that there is a chain of accountability which should be 
identified through such 'second order' policy arrangements within institutions . 

The following recommendations address management for quality. The basis for 
particular recommendations often lies with views put by particular 
stakeholders, particularly those engaged in teaching at a distance and those 
providing support to underpin the delivery of quality distance education. The 
pattern of recommendations has previously been explained and this chapter 
deals with recommendations in the area of policy development, organisation 
and resource management. 

Recommendation 4 - That instit utions encaged in distance education activities 
assign responsibility for distan ce ecfucation to a person at a senior level of 
administrative responsibility. 

WUh increasing devolution of managerial responsibilities to cost centres based 
on academic schools/departments there is a need to ensure that institutional 
goals are not dissipated by the sectional interests of schools/departments. This 
recommendation points to the need to assign responsibility at an institutional 
level for those distance education activities which are appropriately organised 
at an institutional level and for goals which reflect institution wide purposes. 
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Recommendation S - That institutions establish soeciric guidelines for 
disbursement of funds relating to distance education activities, taking account 
of requirements for development fincluding research and course development) 
and delivery (including, besides academic costs, technologv,the academic and 
administrative support .^f stude nts and residential schoolsV 

This recommendation identifies the need to link the planning stage of 
development and delivery issues to funds available for distance education 
activities. Such iinkings should occur at both institutional and 
school/department level, but it is especially important that guidelines exist at 
an institutional level as increasingly delivery issues are related to the provision 
of and access to technologies, which can only be efficiently managed at an 
institutional level. 

Recommendation 6 - That in the allocation of teaching loads, departments or 
schools make ade quate provision for the workload associated with the 
preparation and r evision of course materials, including involvement in peer 

Workload issues have industrial and other implications and responsibilities in 
these areas rest with institutions and heads of schools/departments. 

The allocation of time for preparation of course materials represents a 
commitment of monies in advance of receiving enrolment linked funds. 
Further, the expenditure on preparation is the same regardless of enrolment. 
This difficulty, and the problems associated with allocation of workload 
weightings for the extensive preparation necessaty for quality external course 
materials, has meant that the resolution of what constitutes 'adequate 
provision' is contested within the institution. Nevertheless, the factor of 
adequate workload provision for the preparation and revision of course 
materials is frequently cited as the key quality issue for staff engaged in the 
preparation of teaching and learning materials* This recommendation locates 
the responsibility for allocation at department or school level and assumes that 
the impact of an allocation will flow to institutional management decision 
making levels through normal staff I t^resource allocation bids. 

R f jmmmgndation 7 - That institutions ens ure that students studying in all 
niftd f i ff equitable access to auCTopriate academic and administr ative 

m ppQTt re.g. library, stud v Aills. counselling and computing SfcTVices)> 
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The studies involving academics as stakeholders indicated the importance of 
•student support' for effective distance education. The quality of distance 
learning is a factor of the systems the institution has developed, which enable 
the external student to access support services. The imponance of library 
services as a central factor in the quality of distance learning was mentioned 
by academics. At issue here is the call for institutions to satisfy themselves 
that students, whatever the mode, receive appropriate and equitable levels of 
si^port. 

< 

Recommendation 8 - That institutions p roviding distance education involve 
rtaff with specialist expertise in teaching at a distance in their staff 
development programs. 

It was clear from the survey of services provided by distance education units 
that staff attached to these units frequently provided staff development 
directed towards the improvement of distance teaching and learning. In many 
cases the staff roles were limited to a focus upon instructional design for the 
preparation of course materials, but in others there were staff involved with 
the full gamut of issues associated with teaching and learning at a distance. 
Where institutions also had a specialist unit responsible for the general 
improvement of imiversity teaching/learning and/or research activity, the 
relationship between these 'providers' of staff development expertise was 
unclear. In some instances this was not a problem due to the limited 
instructional design/editorial input role undertaken by staff of a distance 
education unit. However, where institutions sought to move staff development 
in distance teaching and learning beyond materials development issut;:. to 
matters such as delivery systems technologies and their use, or student support 
techniques for external students, there is the need to resolve responsibilities 
and coordinate efforts. 

This recommendation recognises both the desirability of employing relevant 
specialist e)q>ertise %o the area of distance teaching and learning and the need 
to disseminate skills and techniques of distance education within staff 
development programs aimed at the improvement of teaching and learning in 
higher education. 

Recommendation 9 That in thejpl anninf and review of courses, staff with 
specialist expertise in distance education are represented on the Planning or 
review committees that involve e xternal or mixed mode enrolment. 
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This recommendation is aimed at ensuring that academic staff receive 
appropriate advice and support during the planning stage or review of courses 
so that issues surrounding materials production* the use of media and 
communicatioiis technology and the establishment of student support 
mechanisms can be explored and costed prior to production work and teaching* 
The infusion of specialist expertise is recognition that the administrative needs 
and learning environments of external students differ from internal or 
face-to-face students and appropriate recognition of this is a factor impinging 
upon the quality of distance education* Further* at a practical level* the 
involvement of such expertise at a planning phase allows for increasingly 
focused sensitive staff development activities which are 'located* in the 
practical needs of planning or review issues* 

RecommendatioA 10 - That institutions ensure that promotion crit eria or codes 
of practice related to teaching recopiise the quality of contributions to 
distance educaticMi including appropriate scholarship reflected in course 
materials. 

In line with the AVCC statements regarding institutional and departmental 
•valuing' of good practice this recommendation seeks to clarify how 
institutions recognise good practices in teaching and learning at a distance* 
There are often ambiguities in the ways that institutions value the fusion of 
scholarship and teaching within good teaching materials and institutions may 
wish to clarify the 'publication status' of scholarship contained within course 
materials, particularly within the context of promotion and tenure criteria. 

The next chapter looks indicators for monitoring and evaluating the quality 
of distance education. As such indicators are concerned with inputs, processes 
and outcomes there is, inevitably, overlap between the recommendations of 
this chapter and the listing of indicators. Where indicators are broadly 
conceived, the statement of the indicator and any appropriate standard or 
benchmark can often encapsulate the substance of a recommendation aimed at 
quality improvement. Consequently, the next chapter contains a number of 
recommendations aimed at institutions and schools/departments which deal 
with the processes of distance education - equally, some of the 
recommendations of this chapter are reflected in the tabulation of indicators 
under the heading Policy Development and Management. 



Chapter 5 

INDICATORS FOR MONITORING AND EVALUATING 
THE QUALITY OF DISTANCE EDUCATION 

Introduction 

The terms of reference of the Project included the development of 
performance indicators for the area of distance education. Such indicators 
were to cover both the provision of services to support distance education and 
the processes and outcomes of distance teaching and learning, A further term 
of reference directed that the Project's work on performance indicators in the 
area of distance education should take account of current work on performance 
indicators in higher education. 

Thus, the investigation sought to identify appropriate indicators of quality for 
the area of distance education acknowledging the following factors: 

1 the range of performance indicators identified in Performance indicators 
in higher education (Linke et al, 1991), 

2 the framework for the use of indicators within higher education set out in 
Higher education: quality and diversity (Baldwin, 1991). Institutions would 
determine their mission and define what they meant by quality and 
standards of performance against their own objectives - they could also 
select relevant indicators as a means of gauging their success, 

3 distance education involves a partnership of academic teaching in 
discipline areas and institutional siq)port services for academics and 
external students. Both these stakeholders within institutions would need 
to be considered, 

4 the relevance and interpretation of indicators of quality may be different 
for particular stakeholders involved in distance education. 

Performance Indicators in higher educationr4ts relevance to the project 
Our review of Performance indicators in higher education involved looking at 
the purposes of the study and recommendations in relation to the terms of 
reference of the Project. The reports, in dealing with institutional context 
indicators, performance indicators, and indicators relating to participation and 
social equity, provided indicators where data was aggregated by academic 
organisational imits (schools/departments) and area of knowledge 
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(discipline area). The indicators cover all modes of study (unless specific 
qualifiers were identified). The overall stance regarding the application of 
indicators advocated within the report is for their use 'only as an adjunct to 
inform expert judgement, rejecting any prospective application for funding or 
other purposes based on a purely mechanical or formula approach'. 

The Project, in considering data about external mode, chose the usual approach 
of regarding the mode as a subset within commonly used aggregations (eg 
external mode within the unified national system, external mode within 
individual institutions, external mode within schools and departments). When 
considering the recommendations in Performance indicators in higher 
education, any focus upon a single mode as a subset of aggregations presents 
two possibilities for interpretation, namely: 

1 subdivisions? by mode within the levels of aggregation do not cause 
problems with interpretation of the indicators in terms of the original 
purposes and context for which the indicator was meant to apply 

2 subdivisions by mode within the levels of aggregation diston the 
usefulness of the indicator in that mode-specific factors cause 
problems in using the indicator to assist judgements about the 
particular mode in question. 

Given the recency of Performance indicators in higher education it was not 
possible to review recommendations in the light of the above possibilities as we 
lacked an empirical base for deciding between them. The following points 
simply highlight the relevance of Performance indicators in higher education 
and suggest the need for further work in this area: 

1 Two indicators, (academic activity cost per student (Cll) and (total 
recurrent cost per student, (CI 2) have, to some extent, been 
considered in a previous mode specific study on cost relativities 
between internal and external studies, 

2 In the area of performance indicators for teaching and learning it is 
important to note that the recommended indicators are aggregated at 
discipline, AOU or program level. While indicators such as program 
completion rate, mean completion time, student progress rate may 
reveal distinctive results for external load, the report has defined the 
characteristics of the indicators in these areas to have meaning for the 
particular chosen level of aggregation or condition of use. Thus the 
study acknowledges that (program completion rate), (P3) 
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should only be used for full time students in a program of a particular 
size. Interestingly (perceived teaching quality) (PI) may require minor 
adjustment in language and terminology to account for external 
teaching and learning, and, in this form, be applicable at a 'branch of 
learning' level of aggregation. 

In the area of institutional context indicators, the two areas of support 
staff ratio and distribution of recurrent expenditure may be used to 
assist with describing, auditing and evaluating quality of distance 
education. Support staff per number (and/or variety) of courses in 
external mode and support staff dealing with the needs of external 
students per external EFTSU load are possible indicators here. Also, it 
would be possible to look at the distribution of recurrent e)q)enditure 
which services the external load of an institution. While there are 
obvious difficulties in achieving agreement upon how to define and 
categorise expenditure, it is nevertheless useful to explore categories 
such as relative expenditure on infrastructure siqpport for production 
and distribution of course materials and library e)q>enditure tailored to 
meet the needs of external students against the total operating grant. 
However, it is acknowledged that categorisation of expenditure 
depends upon an accepted definition of external load and that overlap 
at the level of infrastructure to support teaching and learning (for 
whatever mode) makes the gathering of data a difficult imdertaking. 

Clearly, this is an important area and reporting the way in which 
external load is allocated resources within institutions would be 
assisted where institutions used such indicators. However, differences 
in structure, resource allocation policy, and historical factors influence 
interpretation of these institutional context indicators and these 
factors would mean that comparisons between institutions could not be 
made on the basis of such indicators. 

Because of the importance of distance education in assisting 
institutions in reaching targets in the area of participation and social 
equity* it is important to highlight any relevant indicators related to 
this area. For example (academic program diversity) (S3) could be used 
to look at the profile of access through external mode thereby assisting 
with reporting upon participation. Here the issue for external 
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provision is one of deciding the appropriate level of aggregation - for 
example, should an indicator for external load apply at a national 
rather than institutional level on the basis that distance teaching 
institutions accept external students on a national (and indeed, 
international) basis? 

There is a need to monitor the use of indicators proposed in Performance 
indicators in higher education and compare the consistency of subset data 
relating to mode specific arrangements to that derived from all modes* 
Institutions should be encouraged to include indicators recommended in 
Performance indicators in higher education within any mode specific 
monitoring and evaluating* 

We now move to those indicators which result from the investigations within 
the Project. 
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How the Project generated incficators relevant 
to jud^ng the quality of distance education 

Since our earlier analysis highlighted the central role of values in any appraisal 
of the quality of education, the Project chose to focxis upon perceptions of 
quality belonging to particular stakeholders* Embedded in these views is a 
complex mix of uses of the term quality - for example, quality could be 
identified as the magnitude of change within a particular student or the 
acceptability of the level of student work arising from the course* That is, 
quality could be both open to comparative judgements (and, this was more 
likely where the learning outcomes of students were subject to wider review as 
in Masters level thesis requirements) and potentially beyond comparative 
judgement (where graduates of different disciplines bring different 
backgrounds to a graduate diploma course and the student outcomes, while 
meeting a minimum requirement, vary considerably)* Other views about quality 
would arise through the role of stakeholders within the education process - 
those involved in the provision of services, for example, saw their efforts to 
support teaching and learning as contributing to quality and consequently saw 
•management for quality* as fundamental to quality improvement* 

Despite these complexities, investigators saw considerable value in 
investigating stakeholder views about quality because they could then ask 
stakeholders to participate in framing indicators based on these views* The 
position of the Project has already been put in Chapter 1 in the section on 
interpreting the Project*s terms of reference* 
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The methodology adopted by the Project was to sample the views of 
stakeholders (those engaged in producing and teaching external courses, 
institutional clients) in an attempt to account for the complex nature of 
judgements about the quality of distance education* As we have already noted 
the terms of reference limited the stakeholders to those above. 

Views from stakeholders were gained through: 

canvassing views of service providers by asking representative groups 
within institutions to nominate indicators which would be useful in 
appraising the quality of distance education. (Appendix 1 contains 
details.) 

a limited number of case studies involving groups of academics 
teaching particular discipline areas. (Appendix 2 contains details.) 
seeking structured responses from institutional clients to cither a draft 
listing of indicators or to questions about factors influencing quality. 
(Appendix 3 contains details.) 

The stakeholders 

The first group of stakeholders, those providing services to academic staff and 
external students are a particularly interesting group as they: 

generally involve both academic and administrative staff. 

provide services to teaching academic staff and are largely regarded as 

^administration* or ^academic administration*. 

are frequently exposed to the tension between professional values and 
bureaucratic e}q)ectations ^ that is, the administrative framework 
within which distance education staff operate may define limits to the 
autonomy of academics in particular areas and the framework is often 
implicitly challenged by academics. 

Consequently, it would be expected that the perceptions of this group would 
involve staff development, instructional design, production management, 
student support and coimselling, and other technical issues arising from 
supporting external students. Any focus upon performance which spans 
academic and bureaucratic values can challenge th^ academic autonomy of 
teachers and the groiq> had particular views about how well others should 
perform as well as how well they could or should perform, given their role in 
the cooperative undertaking of providing services. 
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In addressing the second group of stakeholders, academics who teach at a 
distance, the Project decided to seek views about the quality of distance 
education by studying different discipline areas. There were three major 
reasons for choosing to study different discipline areas. First, as we have seen, 
quality in education (and distance education) is a diffuse concept involving 
multidimensional considerations (Hughes, 1988). It was felt that this 
complexity could be addressed through seeking the collective perceptions of 
the group by asking them to focus upon their activities in providing studies in 
their discipline area to external students. Second, it was considered that issues 
about means (that is, how an area is taught/delivered at a distance) were likely 
to interact with ends (i.e. what is taught, what should be taught) and it would 
be important to allow for such complex interplay. The natural site for this 
interaction is in curriculum and teaching matters and it was thought that such 
matters would best be articulated through considering the course or discipline 
area rather than the concerns of individual teachers. Third, the focus upon a 
particular discipline area may reveal special considerations arising from the 
provision of the discipline itself - that is, particular means or teclmiques 
employed in providing the area to external students might be judged with 
reference to the discipline itself. While there may be generic or imiversal 
factors which impinge upon quality in education at a distance, there may also 
be criteria used to judge quality which are derived from an understanding of 
the particular discipline. 

In seeking perceptions of academic staff regarding what constitutes quality in 
distance education and what mechanisms may foster quality improvement, the 
Project used a case study approach which sought to identify beliefs which 
revealed consensus or contestation within the groiq). 

The Project investigators had hoped for a number of studies which would cover 
a range of discipline areas, a range of level of awards, and a range of delivery 
techniques. While the studies presented by the Project are useful and provide 
•indicators' of quality in distance teaching and learning it is clear that there is 
need for further extensive work in recording practitioner theories and 
practices in the area of quality improvement of distance teaching and learning. 

The third stakeholder, institutional clients, presented a range of views about 
quality often from distinctive and imique perspectives associated with both 
what they would regard as 'client satisfaction' with the provider (Distance 
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Education Centres) and the projected needs of their students. Details of the 
ways in which information was collected from this heterogeneous group are 
contained in appendix 3. 

Indicatcx^ of the quality of distance education 

The authors have drawn together the views of these stakeholders in a listing of 
indicators in an attempt to cover the diverse activities of distance education. 

As we have already noted a listing of indicators compiled in this way has 
shortcomings which signal the need for further work. First, the groups sampled 
did not cover the possible range of types of courses and mix of media with the 
possible result that what they emphasise is not universally applicable. Second, 
there are differences in emphasis among stakeholder groups. Third, relying on 
the views of current distance education practitionen; gives credibility to 
current practice and voice to assumptions that m?.y not be warranted over the 
long term or in all contexts in which distance education is practiced. 

The indicators have been organised under policy development, organisation and 
resource management (policy development and management, service provision, 
staff development) and processes of distance education. The following 
tabulation displays indicators providing information about the area under 
consideration, their applicability according to 'level of aggregation* and a 
statement about standards or benchmarks. 
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Policy development an d manag ement 



Area under consideration 



Applicability 



Standard/benchmark 
or elaboration 



Policy framework for 
distance education 
within an institution 



institutions 



existence of 
specific guidelines 
for disbursement of 
funds related to 
distance education 
activities 
existence of a 
consolidated listing of 
policies related to 
distance education 



Reporting achievements 
and developments in 
distance education 



institutions 



detail within annual 

reports of institutions 

(or similar) of 

developments in: 
external course 
development and 
production 

• technologies employed 
in distance teaching 
and learning 

• research and 
evaluation of distance 
education 



3 Acknowledgement of at institutional . existence of 

expertise in distance level institutional approval 

education within 'code of practice' 

promotion/tenure in relation to 

criteria teaching and 

appropriate 
recognition by this 
code of the practices 
of distance teaching 
and learning 
application of the 
above criteria within 
selection processes 
related to 
promotion/tenure 
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Area under consideration Applicability Standard/benduoark 

or Elaborattcm 



Provision of release 
time for academic staff 
engaged in course 
writing and peer review 



provision within 
schools/departments 



appropriate levels of 
support- eg 1/3 to 1/2 
load release per semester 
subject equivalent to 
1/6 full time student 
load 
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Staff development 

Area under consideration Applicability Standard/benchmark 

or elaboration 



1 conduct of professional institution or 
development programs distance 
for academic staff education unit 



formal document outlining 
the strategy for professional 
development to improve distance 
education available 



conduct of 
staff awareness 
program in relation 
to institutional 
distance education 
policy 



institution and 
distance 
education unit 



existence of specialist 
induction program for new 
staff 

recognition of this function 
within the role statement 
of a staff member of the 
institution 



access to expertise/ provision within • identified program 

advice on materials institution of staff development 

design directed towards materials 

design 

cotirse team membership 
of persons with specialist 
expertise in distance education 
during planning/development 
stages 

availability of institutional 
'guides* to materials design 



access to expertise/ 
advice on instructional 
media 



provision within 
institution 



identified formal 
liaison mechanism 
between distance education 
unit and relevant service 
provider as a service 
provided by the distance 
education unit 
identified mechanism 
regarding approval of 
costs related to employing 
instructional media 
within the delivery 
process 



5 access to expertise/ provision within . as above 

advice on the use of institution 
computer/communication 
technologies employed 
for external delivery 
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Service prevision 
Applicability 



Standard/benchmark 
or elaboraticMi 



1 Institutional 

responsiveness to 
student enquiries 



distance 
education unit 



existence of a published 
statement about the 
response time to contact 
students 



Receipt of learning 
materials 



distance 
education unit 



Timely pre-preparation 
of materials by academic 
staff 



schools/departments 



number of late 
starting students/subjects 
attached to 
schools/departments 



published guidelines 
to staff on schedules 
for production 
existence of targets 
related to the 
percentage of the total 
of course materials 
which are prepackaged 
in advance of the 
semester in which they 
are offered 



Technical quality 
of learning materials 



distance 
education unit/ 
Printing facility 



number of student and 
staff complaints 
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Processes of distance education 



Area under consideration 



Applicability 



Standard/benchmark 
or Elaboration 



Availability and use 
of preparatory materials 
to assist students in 
their role as an 
external student - 
generalised information 
related to the 
distance learner 



distance 
education unit 
/Study Skills 
Coordinator/ 
Student Support 
Services 



existence of specific 
support mechanisms eg 
*taster units* 
counselling contact 
prior to course commence- 
ment available where 
needed 



Review of instructional 
blueprints* by expert 
panel against academic 
and distance teaching 
criteria 



Schools/ 

departments with 

distance 

education 

professional 

staff 



adequate representation 
of subject e^q>ertise 
for content validation 
adequate representation 
of specialist e3q>ertise 
in distance education 
review processes carried 
out against agreed 
criteria 



Study materials and 
processes: 

. account for student 

background 
« appropriate materials 

and learning 
. interactions 



Schools/ 

departments and 
distance 
education unit 



existence of: 

clearly defined minimum 
provision of a range of 
clear starting points 
dependent upon student 
backgroiuids 
assistance to students 
in clarifying their own 
backgroimd knowledge and 
skills 

enunciation of realistic 
learning objectives a 
identify and» where 
appropriate, provide 
necessary resource 
material 

appropriate breadth and 
depth of learning 
activities to challenge 
each student intellectually 

, provision of c^>portunities 
for interaction with 
staff and other students 

« provision of academic 
.siq:»port in regular and 
informative assessment 
and self assessment 
demonstrable linking of 
assessment with the 
objectives of the topic 
of stucfy 
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Area under consideration Applicability Standard/benchmark 

or elaboration 



4 Perceived teaching schools/departments Possible trial use of 

^"^^i^y Course Experience 

Questionnaire - Indicator 
PI Linke(1991)p58 



5 Variety of interactions schools/departments During the period of 

student study the use of 
one or more of: 
. phone contact with 
individual students 

* audio/video 
conferencing 

of groups of students 

* fax» Email computer 
conferencing 

* extended 
correspondence; 
journals 

on-campuis schools/ 
workshops 



6 Lecturer availability 
to students 



schools/departments 



notification to 
students of a variety 
of contact mechanisms 
notification to 
students of times 
that a relevant staff 
member is available 
to deal with academic 
matters 



7 Assignment marking schools/departments , existence of a 

published norm for 
turn around time for 
assignments 

. existence of guidelines 
for marker feedback to 
students including the 
incorporation of 
teaching objectives 
within the response to 
assignments 

• existence of policy 
regarding flexibility in 
extensions to 
submission dates 
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Area under consideraticm Applicability Standard/benchmaric 

or elaboration 



8 Student feedback institution, . existence of 

on the quality of distance questionnaire or 

student support and education unit, monitoring mechanism 

• administrative services Library to enable collection 

supplied by the of inf ormation on 

institution student satisfaction 

with learning siipport 



Monitoring student 
progress and achieve- 
ment 

• student progress rate 



schools/departments 



application, on a trial 
basis, of the use of 
Student Progess Rate « 
passed subject 
load/ certified subject 
load-see Linke (1991) p70. 



Discussion 

We have already noted that the methodology used to generate the above 
indicators has both strengths and weaknesses. However, as we have already 
briefly noted, the recent Draft Code of Practice for University Teaching 
(AVCC, 1991) reflects many of the issues raised. 

The Draft Code, like the listing of indicators provided for Quality and 
Standards in Distance Education, covers the role of the institution, the 
department and individual teachers. Parallels exist in the areas of recognition 
for teaching excellence, the existence of professional development programs, 
the provision of resources so that teaching may be conducted as a scholarly 
activity, the provision of time and resources for staff to develop subjects and 
teaching materials, the need for monitoring and review of subjects and 
teaching materials, and the characteristics of good teaching, assessment, and 
efforts necessary to promote student learning. While the draft code does not 
make explicit ir* Aon of external mode delivery or the needs of external 
students it nevertheless highlights characteristics of good teaching and 
learning which, for distance education, are evident in study materials and 
processes employed in distance education, and the support services provided by 
the institution. The indicators of quality and standards of distance education 
generally focus upon particular asp>ects of processes relevant to distance 
education and can generally be linked with a broader statement about quality 
within the draft code* 

The indicators identified by the study require a further phase of specification 
and elaboration (especially in deciding appropriate standards or benchmarks) 
prior to their application in assisting judgements about the quality of distance 
education within a particular context. Given this situation there is clearly the 
need for further work to monitor the ways in which indicators such as those 
listed (and others) are used in practice and to evaluate whether such indicators 
are indeed useful in assisting judgements about quality. 

Further Recommendations 

As we have already noted in chapter 4, recommendations about quality 
improvement are linked to the indicators identified in the particular areas and 
this next section outlines recommendations about the. processes of distance 
education. 
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Recommendation 11 - That institutions consolidate and make publicly available 
their specific policies that deal with arrangements for off campus provision . 
Any review of an institutions' policies, goals, and targets for external load will 
be facilitated by a coordinated and consolidated policy framework which has 
been approved by the governing body of the institution. Institutions designated 
as Distance Education Centres might model the production of such policy 
frameworks, and a published and consolidated listing of policies may assist 
policy development in other higher education institutions* 

Recommendation 12 - That ins titutions establish policies for the development, 
implementation and resourcing of the use of appropriate technologies in 
distance education^ 

Many academic staff report that institutions are unable to provide technologies 
and support necessary to implement their needs for distance teaching and 
learning in their area. For example, staff teaching external students often 
request desktop computing facilities, connection to AARNet, and support for 
E-mail systems (which would enable students who have access to personal 
computers to carry out interactive communications) only to find that 
institutions are not able to provide adequate access to such technology for 
staff or support such uses of technology with technical advice and back-iq) 
service. At the basic level of audio conferencing some institutions have low 
levels of support in assisting with the use of teleconferencing (either 
administrative or technical) thereby effectively discoiiraging the use of the 
technology. It is ironic that a key area of innovation and growth within 
distance teaching and learning is in the application of computer based 
technologies (with production technologies such as desktop publishing systems, 
video production etc and communication technologies such as E-mail, video 
conferencing etc) yet the operational imits within Distance Education Centres 
which provide support for distance teaching and learning are not resourced to 
move into using technologies extensively. There is a major need for Distance 
Education Centres to develop infrastructures which enable the efficient use of 
communication and information technologies within higher education, 
particularly distance education and open learning. This recommendation is 
directed towards starting processes which are aimed at entrenching the use of 
such technologies within normal delivery processes of distance and open 
education. 
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Recommendation 13- That institutions revie w anv policy statements on student 
rights and responsibUities to ensure their appro priateness to external students. 



Some institutions have published a 'Charter of student rights and 
responsibilities' which deal with a range of academic and administrative 
matters. 

The idea of a Student Charter has been taken up in Quality tuition: optimum 
academe support for all students (1990) a consultative document prepared by a 
range of 'stakeholders' in the quality analysis of the British Open University. 
In the view of this review group such a charter would be distributed to every 
student upon registration in a course, and would cover course materials, 
assessment, administration, all forms of academic support, counselling, 
feedback, financial assistance etc. Under these headings 'information should 
be given about the student's rights and entitlements, together with a carefully 
worded indication of what students are not entitled to, but may sometimes 
expect'. (pl8) 

This recommendation acknowledges the need to provide information which 
covers the circumstances of external students. 

Recomme nriation H - That procedures for planning and developing distance 
education, and monitoring and reviewing performance in distance education, in 
the area of course ma terials and their implementation address : 

format and presentational quality of learmng materials: 

instructional quality including the use of appropriate teaching and 

interactive processes: 

workload and assessment load: 

arrangements for st atent ffunWTt 

The information gathered relating to 'nominated indicators' supports the 
importance of the factors identified in judgements about the quality of course 
materials and their implementation. Further, materials produced by Distance 
Education Centres or other higher education institutions in conjunction with 
Distance Education Centres may have national and international use if 
purchased and purchasers' e)q)ectations regarding appropriate quality should be 
met by the system. Institutions would be expected to detail appropriate 
benchmarks for the above areas. 
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Recommendation IS - That institutions with a substantial involvement to 
distance education include in their annual reports maior achievements or 
developments in the areas of: 

external course development and production 

technolopes used in di stance teaching and learning 

student support techniques 

research and evaluation of distance education 

Dissemination of information about centres of excellence in distance education 
is useful for clients of the system - students, other higher education 
institutions, and corporate bodies. The publication of such information in a 
consolidated form with other contextual information about the institution will 
enable interested parties to effectively gauge the strengths and special 
interests of particular Distance Education Centres. 



ition 16 That institutions involved m distance education identify 
and publis h a list of input, process and output indicators that thev intend to use 
in monitorinE and reviewmg performance to distance education against 
relevant institutional objectives . 



Notwithstanding the central role of institutional self monitoring, it is 
important that Distance Education Centres are able to clearly identify their 
quality improvement role in the area of distance education. Other higher 
education institutions, as clients of the Distance Education Centres, should be 
aware of the quality assurance processes that Distance Education Centres have 
in place. 

Recommendation 17 - That all c ourse materials used to distance education 
carrv a statement of acknowledg ement for input and support received from 
ii^ividuals and organisations involved to their development and production . 

This recommendation is directed at providing information to assist clients with 
appraising the quality of course materials by identifying various inputs. 
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Chapter 6 

REVIEW AND FUTURE DEVELOPMENTS 



Review 



The project to investigate quality and standards in distance education has 
attempted to address its terms of reference against a background of change in 
relation to the purposes of Distance Education Centres within the higher 
education system and the developments in the agenda for quality in higher 
education. 

The timing of the project was significant in that: 



its terms of reference are likely to have been more restricted had they 
been developed after the release of Quality and diversity and the HEC 
disciassion paper. 

during the project there was further questioning of both the role of 
Distance Education Centres and the means by which *(l:.*tance education* 
might be identified. Debate about a multiplicity of modes and methods 
incorporating various aspects of Mistance mode* and open learning as 
rival/complementary concepts served to highlight difficulties in 
identifying factors influencing the quality of distance education, 
the project sought to build upon current work on performance indicators 
and awaited publication of reports in this area. 



A further factor influencing progress with the investigation was the diversity 
of systems and procedures within institutions designated as Distance Education 
Centres and the instability of the management and structures supporting 
distance education due to reviews and changes brought about by amalgamations. 



Early in the project the reference group questioned the omission of 'student 
perceptions of quality* from the terms of reference. It became clear that the 
way that the terms of reference had limited the stakeholders to 'providers* of 
distance education (with the exception of corporate clients) implied constraints 
upon the way in which the project would view quality - for example, if quality 
of education was to be linked to Value added* in the student then the omission 
of a student perception of quality (or a perspective of quality framed from the 
student attributes which signify excellence) was important. The investigators 
recognised this feature in the terms of reference and. given the scope and 
complexity of the existing terms of reference, took the view that to include 
students as stakeholders would necessarily involve a larger and more complex 
ERXC ^^^^ already proposed. 
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The investigation could not be considered a research study about factors 
impacting upon the quality of distance education. Because data was collected 
at different institutions, the investigation employed different methodologies 
depending upon the context of participants. Drawing together such data and 
proposing generalisations is a difficult undertaking. To assist the reader in 
evaluating our interpretation of the information from the various sources we 
have provided extensive appendix material containing 'raw' data. 

A further area of concern was the limitations imposed by the practical 
problems of conducting case studies in relevant and different areas involving 
varied educational techniques. The investigation was not dble to explore 
quality factors arising from approaches which employ technologically advanced 
delivery systems. Consequently a case could be made that the 
recommendations arise from, and are directed towards imjjroving, a model of 
distance education representing an augmented form of correspondence 
education. 

Nevertheless, the investigation has provided a range of recommendations 
which, if addressed, would have a significant impact upon institutional 
practices. The thrust of the report is to provide recommendations and 
indicators which might form the basis of any institutional review of its 
procedures which ensure quality distance education. Clearly, the materials of 
the report would require integration into the overall institutional strategy and 
should not be considered in isolation. This is particularly evident where 
recommendations are addressed to sections of the academic structure of 
institutions and therefore involvG other considerations for successful 
implementation. 

If the success of the report is to be gauged by the extent to which institutions 
employ the listed indicators or engage with the substantive issues in the 
recommendations, issues about the appropriateness of indicators, the costs of 
collecting information, and the extent to which the indicators assist evaluative 
judgements will be central. Such practical considerations have meant that the 
suggestions about standards or benchmarks have been broadly framed so that 
institutions can insert their own fine-structure depending ^pon context, cost, 
stage of development of quality considerations etc. Because of this planned 
flexibility there is a need to monitor and refine particular indicators and 
research both their intended and unintended effects upon quality in distance 
education. 

( o 
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Future developments 

At a time when many Universities are developing policy approaches to 
addressing quality within their institution it would seem self evident that 
institutional responses would cover particular areas identified in this report. 

Recommendation 18 - That NDEC monitor institutional responses to the 
recommendations of this report bv seeking sp ecific information on particular 
areas covered in the r ecommendations . 

This recommendation acknowledges the need for an expert body such as NDEC 
to look at the ways that institutions assure quality in distance education. 
Clearly different institutions will have different priorities in relation to their 
educational programs and practical considerations will mean that institutions 
will determine what they mean by quality and standards in terms of the 
policies and procedures that they develop. Consequently, any assessment of 
the quality of distance education within the higher education system will 
require the monitoring of the areas of information addressed and the methods 
of arriving at an assessment of quality considered in relation to declared 
benchmarks or standards. 

There are a number of possible developments which would assist work in 
appraising the quality of distance education. 

1 There should be consideration given to funding a developmental project 
which attempts to report on ways in which the recommendations of this 
project were addressed within a particular institiitional context. Such a 
study may well consider broad institutional concerns as well as changes 
which follow at a departmental/school/discipline level. 

2 There should be consideration given to funding studies which focus upon 
the lise of delivery technologies within distance education and the groimds 
for claims that the xise of such technologies improves the quality of 
distance education. In particular, consideration should be given to 
addressing a range of issues arising from introducing new technologies - 
administrative and technical support, teaching capacities, implications for 
access to the technology and equity issues, the means of evaluating 
effects through student feedback and other mechanisms, the unintended 
consequences of the use of such technology etc. 



Appendix 1 

INDICATORS OF QU ALR Y OF DISTANCE EDUCATION 
THE VIEW OF STAFF FORMALLY ATTACHED 
TO DISTANCE EDUCATION CENTRES 

Introduction 

This appendix considers the views of one stakeholder - those staff that are 
formally involved in supporting the efforts of those teaching at a distance. In 
the majority of cases such staff belong to the Distance Education Centre (or 
Distance Education Resource Centre, Open Learning Institute or similar) and 
are those who advise, manage, or provide technical and administrative services 
to support distance teachers or external students. 

The methodology employed by the Project was to ask institutions designated as 
Distance Education Centres to assemble a representative group of staff* Staff 
in this group were asked to interpret their individual roles in ways which 
represented an ideal situation in relation to the provision of quality services 
which might result in excellence in education at a distance* The instructions 
to conduct the exercise dealt with its purposes, ways of getting started, the 
characteristics of indicators, ways of refining statements, and what would 
happen to the indicators that were nominated. Details of the approach are 
provided at the end of this Appendix. 

The returns from the four institutions involved in this part of the project were 
compiled and re-circulated to institutions requesting that areas be reviewed in 
the light of the compiled information and, if necessary, the initial materials 
provided be changed in the light of this review. The following presents the 
nominated indicators - note that there are areas of overlap representing 
different approaches to the same area. 
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Arta under consideration 



Indicator or elaborative statement 



1 Policy framework for distance 
education %rithin an institution 



existence of explicit written policies 
evidence of implementation and resourcing of policies 
staff awareness of institutional distance education 
pol icy 



2 Academic and support staff 
. policies for appointment and 
professional development of all 
staff in relation to necessary 
specialist skills required for 
distance education 
. reward structure and career paths. 



existence of professional development 
programs in distance education 
evidence in promotion criteria of recognition 
of excellence in distance teaching 
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. staff development of academic 


access to adequate expertise and media to facilitate 


teaching staff 


production of materials, which include: 




- AV and electronic 




- editorial, print, graphics 




- design and evaluation 




- reproduction 




- provision of release time for academic staff writers 




increased use of non-traditional media for DE/open 




learning delivery 




. increased student support through use of communication 




technologies and study centres 


3 Neeting market needs 


evidence of courses responding to the needs of 




'community groups* 


4 Research and development in 




distance education 




. in relation to students 


. existence of an inbuilt program for staff to 


. in relation to teaching and 


monitor programs 


1 earning 


adequate resources allocated for research and 


. in relation to policy 


development 


development and implementation 


rates of success of staff in attracting external 




funding for research and development in distance 




education. 


5 Enrolment 




. refusal of student admissions 


ratio between qualified students admitted and students 




refused entry. 


6 Presentation of learning materials 




. technical quality 


number of complaints 


7 Student preparation for distance learning 


. availability of preparatory 


inventory of materials and face to face 


materials to assist students 


contact given prior to course commencement 


with distance learning 




8 Receipt of learning materials 




. availability to student 


number of late starting students/subjects 


. dispatch of learning materials to 


students receive materials on confinnation of 


students 


enrolment or by the week prior to semester commencing 




. academic staff are made aware of delivery services 




and deadlines for these in their induction 


BEST COPY AVAILABLE 


standard materials are delivered in one or at most 


two dispatches; other dispatches are for incidentals 
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Ar«« undtr contidtration 



Indicator or daborative sUtenent 



Study materials and proctssts 

. accounting for student background 
. appropriate materials and learning 
V interactions 



standards of production 

for printed materials 

timely pre-preparation of materials 

by academic staff 



existence of. 

. dearly defined minimum entry competencies 

. provision of a range of clear starting points dependent 

upon student backgrounds 
. assistance to students in clarifying their own 

background knowledge and skills 
. enunciation of realistic learning objectives 
. identify and, w^«ere appropriate, provide necessary 

resource material 
. appropriate breadth and depth of learning activities to 

challenge each student intellectually 
. provision of opportunities for interaction with staff 

and other students 
. provision of academic support in regular and informative 

assessment and self assessment 
. demonstrable linking of assessment with the objectives 

of the topic of study 
. publication of a manual of national standards 

for production of print and non*print OE materials 
. materials for SOX of subjects pre-packed in advance of 

semester in which they are offered 



10 Design and developnent of a sample 
module 

. facilitation of more efficient 
development, writing and production 
of a new unit. 



Record instructional designer, education officer and 
production time spent on each new unit for a semester, and 
compare total time spent on units with sample module to 
time spent on those without sample module. 



Lecturer acceptance of, and support for, the 'sample 
module* approach. 

Compare the instructional strategies, design and layout of 
subsequent modules with the sample module. 



Honitor feedback, from student evaluation questionnaires, 
on the unit's instructional quality. 



11 Design and development of learning Instructional bluenrint cf each unit is well documented, 

packages through use of checklist. 

. assistance for subject matter 
specialists engaged in distance, mixed Instructional blueprints are assessed as high quality by 
mode and continuing education an expert panel, 

activities in instructional materials 

design and development Sample snodules are produced and reviewed prior to further 

unit development. 

Unit development and production meet deadlines. 

Evaluation findings* including student performance and 
student feedback » impact on subsequent design and 
development of unit. 

Number of staff parti ciu^ating in instructional design 
workshops and seminars. 

Workshop feedback, formal and informal, is monitored. 
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Are* under consideration Indicator or elaborative statement 

12 Print production cycle 

. provision of efficient processes for Record the number of changes to the pre-production 
managing the degree of change to information, 
study materials, hence reducing work- 
load and materials wastage. Record the number of changes to the database during the 

time leading cp to the production of the materials. 

Monitor the number of pieces in each category, eg 'New*. 
*Major', *Minor*, 'No change'. 

Estimate the approximate number of hours' work required 
for the next production period. 



Monitor the history of a unit showing its adherence to the 
production cycle or otherwise. 



13 


Student support 


. evidence of institutional structures, resources and 






procedures to enable criteria of immediacy, accuracy. 






consistency, completeness and appropriateness in student 






support to be met 






. evidence of student satisfaction with support services 






provided 




student support and open learning 






shared use with DETAFE of study centres 


. increased enrolments in rural areas 




range of technologies used 


. lowered attrition rate 




national credit transfer bank 


. increased participation by under-represented groups 




flexible administrative policies 


. increase in number of courses/subjects offered by OE 


• 


further development of TV 






OL Project 
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Institutional r«sponsiveness-student 






enquiries 


Monitor traffic in and out and response times for 






enquiries . 






Record statistics in comparison to previous periods of 






time. 






Modify procedures to counteract 'bottlenecks' to solution 






of enquiries. 




requests for information responses 






to enquiries, liaison arising from 






student contact 


services within 24 hours 


15 


Student assignments 






marking turn around 


maximum of 10 days from receipt of assignment 




Processing of student assignments 


. same day turnaround of received assignments 




within the DEC and by academic staff. 


. return to students within 2 weeks of receipt 




marker feedback 


minimum of 50 pertinent, succinct and encouraging words 






per assignment 




marker reliability 


analysis of variance for difference in marks given by 






different markers 



Arta uiKier considtration 



Indicator or tiaborative stat«ncnt 



16 Student conplttion 

. attrition rate ratio of overall commencement and overall completion 

time taken to complete average number of active semesters from commencement to 

completion 

continuation rate 
. graduation rate 
. career success 



17 Copyright 

. provide external students with ready Monitor the number of copyright applications per semester, 
access to existing learning resources 

from a varietv of sources. Record the number of items submitted without complete 

documentation. 

To meet the requirements of the copyright 

act and of the CM agre«?flent as it Record time spent on copyright processing by Education 

relates to study material. Officers and copyright clerk (there may be an increase in 

time spent on advice to writers earlier). 

To ensure DEC staff receive adequate 

bibliographic information wnen copyright Monitor number and total amount of copyright fees paid per 
material is submitted by writers. se^<ester. 

To ensure DEC has adequate time to Monitor number of copyright disputes with publishers, 

process copyright applications and 

receive responses before study material Monitor number of pieces withheld from printery, or with 
is due to be printed. copyright material removed, due to copyright delays. 

To minimise the cost to DEC of payments Undertake random copyright checks on individual units to 
incurred by the production of copyright ensure that legal requirements are satisfied, 
material . 
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A group from another Distance Education Centre, not involved in nominating 
indicators, reviewed the listing and provided the following comments or 
additional statements. 

Area under ccmsideration Comment or additional 

statement(s) 



1 Policy framework for distance 

education within an institution Accept 

2 Academic and support staff Add: 

existence of appropriate 
professional (distance education) 
staff in Distance Education 
Centre. 



3 Meeting market needs 

4 Research and development in 
distance education 

5 Enrolment 

6 Presentation of learning materials 



Add: 

relevant and 'up-to-date* 
content and technology. 



Accept 

General concern was expressed 
about this statement given: 

(i) Present over enrolment; 

(ii) Special entry provisions 
under equity programs. 

Accept 



7 Student preparation for distance 
learning 



Some concern expressed about 
'face-to-face contact'. Does 
this include video confere^mcing 
or other appropriate media 
contact? 



8 Receipt of learning materials 

9 Study materials and processes 



Accept 
Add: 

Provision of student support of 
interaction 



10 Design and development of a sample 
module 



General concern e^ressed about 
the time spent in recording 
details for each new subject 
given the differences which may 
occur such as: 

(i) Subject only in print mode 
(eg Literature) as against a 
multi-«media production of 
an Environmental Science 
subject; 
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Area uzider consideration 



11 Design and development of 
learning packages 

12 Print production cycle 



13 Student support 



14 Institutional responsiveness- 
student enquiries 

15 Student assignments 
• marker feedback 



. marker reliability 



16 Student completion 



17 Copyright 
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Comment or additional 
statement(s) 



(ii) Difficulty of content; 

(iii) Experience of Subject 
Author(s), First external 
subject or many previous 
external subjects. 

The general concern expressed was 
what will the indicator achieve? 

Accept 



The comment here was that this 
was of less importance than most 
of the other indicators. The 
number of changes may indicate a 
necessity to be *up-to-date' or 
required because of changed 
circumstances. 



Add: 



Counselling services 
appropriate for distance 
education students . (This 
was seen as not necessarily 
indicated in the first 
statement for this section.) 



Accept 



Concern was e)q>ressed about this 
indicator. 

Additional comment was: 

(i) increasing awareness by 
markers for students by 
providing marking keys, 
evalimtion criteria as well as 
encouragement. 

Monitor the difference in marks 
given by different markers. (There 
are a variety of measures that 
could be applied.) 

Some concern expressed over this 
whole section in terms of what the 
indicators achieve. 
How would career success be 
measured in terms of studies? 

General comment indicated that 
this was an imimportant indicator. 
It was an institutional necessity and 
may have economic parameters (eg, 
try to reduce costs) but was not 
seen as a suitable performance 
indicator. 
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Thus, there was general support for the indicators with the following 
excepdons: 

enrolment - the points made by the review group are important in the 
context of the linking of access and equity arrangements with 
distance education* Matters dealing with characteristics of the 
student enrolment have been considered in Performance indicators in 
higher education (Linke et aL 1991) and covered under Student 
Preference Ratio, Student Application Rates, Student Offer Ratio, 
Average Student Entry Score* 

design and development of a sample module - the use of 'time spent* 
as an indicator is open to challenge because of the impact of context. 
However, there are other elaborative areas such as 'lecturer 
acceptance' and monitoring student feedback. 

print production cycle/copyright - these are seen to be less important 
and matters which impact upon efficiency 

student completion - matters dealing with student completion rates 
have been fully discussed in Performance indicators in higher 
education. Because student completion indicators need to be 
ag^egated at AOU or academic program levels such indicators and 
their interpretation are a matter for AOUs. 

Consequently, for staff who advise, manage or provide technical and 
administrative services to support distance teachers or external students there 
seems to be acceptance of areas 1,2,4,6,8,9,11,13,14,15, but various 
qualifications about the general applicability of indicators attached to these 
areas. 

An initial attempt by the project investigators to reH3rganise and simplify the 
indicators resulted in the following tabulation which divides indicators 
according to 'inputs (In), processes (P) and outcomes (O). It was felt that this 
form of organisation might assist institutions in their choice of relevant 
indicators. 



Indicator 
statement 



-84- 
Applicability 



Standard/ 
benchmark 



In 1 access to expertise/ 
advice on materials 
design 



provision 

within 

institution 



In 2 access to expertise/ 
advice on instruct- 
ional media, particularly 
audiovisual 



provision 
within institution 



formal 

liaison between 
DEC and relevant 
provider as a 
service provided 
by the DEC 



In 3 access to expertise/ 
advice on the use of 
computer/communication 
technologies employed 
for external delivery 



provision within 
institution 



In 4 provision of release 
time for academic 
staff engaged in 
writing course 



In 5 earmarked monies 
from institutional 
sources for research, 
development, quality 
monitoring of distance 
education 



provision within 
Academic Organisa- 
sation Unit (AOU) 



institution 



various levels of 
support - approx. 
1/3-1/2 release 
per subject 
equivalent to 1/6 
full time student 
load 



PI conduct of staff 

awareness program 
in relation to 
distance education 
policy 



institution 
and DEC 



specialist 

induction 

program for new 

staff 

formal 

recognition of 
the function 
within the role 
statement of a 
staff member of 
the institution 
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Indicator 
statement 



Applicability 



P2 conduct of 
professional 
development programs 
for academic staff 



DEC 



Standard/ 
benchmark 



formal strategy 
document for 
professional 
development 
available 



P3 Availability and 

use of preparatory 
materials to 
assist students in 
their role as an 
external student - 
generalised 
information related 
to distance learner 



DEC 
/Study 

Skills/Student 
Support Services 



face-to-face 
contact prior 
to course 
commencement 
simulated 
learning and 
support 
exercises 
through pre- 
paratory 
materials 



P4 process to ensure DEC 

that academic staff /Production 
are made aware of Services 
production and 
dispatch deadlines 



induction program 
regular 
bulletins to 
contact staff 



P5 review of 

'instructional 
blueprints' by expert 
panel 



DEC 

/Academic 

Organisation 

Units 



adequate 
representation 
of subject and 
instructional 
expertise 
review against 
e3q>licit /and 
agreed <friteria 



P6 decisions arising from 
student feedback 
to be incorporated 
into redesign/redevelop- 
ment of materials* 



DEC 

/Academic 
Organisation Unit 



01 existence of 

e^licit written 
policies 



institution 
and DEC 



formal acceptance 
of policies by 
governing body of 
the institution 
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Indicator 
statement 
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Applicability 



Standard/ 
benchmark 



02 existence of 

explicit resourcing 
policies related to 
the external load 
of the institution 



institution 



formal acceptance 
of policies by 
governing body 
DEC or DERC 
operations 
designated a 
cost centre 



03 evidence in 

promotion criteria 
of recognition of 
excellence in 
distance teaching 



institution 



04 success of staff 
in attracting 
external funding 
for research/ 
development/ 
consultancies in 
distance education 



institution 



rates of success 
and total grant 
monies received 



OS technical DEC 

quality of learning /Printing 
materials Facility 



number of 
student and 
staff complaints 



06 availability to 

student of learning 
materials prior to 
the start of the 
semester 



DEC 

/Academic 
Organisation Units 



numbers of 
late starting 
students within 
AOU areas 



07 existence of a 
published turn 
around time for 
assignments 
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DEC 



08 existence of a 

published statement 
about the response 
time for an enquiry 
involving services 
provided by DEC 
support staff 



DEC 



. published 
standard or norm 
for institution, 
or 

. published standard 
or norm for 
Academic 
Organisation 
Unit 



Indicator 
statement 



Applicability 



Standard/ 
benchmark 



consideration of 
attrition data for 
external load taken 
by Academic 
Organisation Unit as 
part of the review 
process for the AOU 



DEC 

/Academic 
Organisation Unit 
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Appeiidix2 

QUALITY OF DISTANCE EDUCATION 

VIEWS OF THOSE WHO TEACH AT A DISTANCE 

WITHIN PARTICULAR DISCIPLINE AREAS 

Introduction 

Studying academics* perceptions of the quality of distance education in 
particular discipline areas is a complex undertaking involving the identification 
of perceptions (i.e. understandings, comprehensions, feelings, viewpoints) about 
factors influencing the quality of teaching and learning at a distance* Through 
a number of similar studies in different discipline areas or with specific groups 
of staff (e.g. those employing specialist delivery technologies in programs) the 
investigators hoped to construct a larger, albeit limited, picture of how 
academics see quality in distance education. The collated results of such 
studies may well reveal general and specific features of quality and point to a 
number of possible generic and specific indicators which might be appropriate 
for use by academic staff when describing or appraising issues about quality in 
distance education within their area. 



Participating institutions were provided with a case study on academics* 
perceptions of quality in distance education as a possible methodology which 
might be employed when conducting an investigation with this group of 
stakeholders. (Case Study 1). A modified form of this approach was used in 
Case Study 3, while a group approach to nominating key indicators was used in 
another institution (Nomination Study 2). 

Case study 1: Report and summary of work undertaken 

This study involved a group of six academics teaching post graduate awards in 
religion and religion education. There has been strong demand for their 
courses by interstate students and staff have national reputations as 
teachers-at-a-distance, and national and international reputations as 
academics. After individual interviews with staff, edited transcripts were 
circulated to all participating in the case study. Three group sessions followed 
where staff explored various issues raised in the transcripts. 



To illustrate some of the issues which arise from such a case study, it is 
interesting to provide an outline of the results and a summary of some 
impressions that were gained. 
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The Vesults' of the case study can be summarised in the form of an outline of 
the indicator areas which arose from the group discussion. Four areas were 
identified and it was believed that if the matters contained within the 
elaborative statements about aspects of these areas were addressed, any 
changes would influence the quality of teaching and learning at a distance. 
The structure for the areas is derived from a ^transmission model* of pedagog>' 
and represents an educational position broadly held by the group. Briefly, the 
structure can be outlined in the following way 

Area 1: General institutional siqiport mechanisms for teachers and students 
access to staff development concerning teaching at a distance 
workload and other acknowledgement for materials preparation 
access to technology used for distance teaching and learning 
support for students provided by the Distance Education Centre 
institutional support - library services 

acknowledgement of ability/familiarity with studying externally 

Area 2: Preliminaries, to course delivery - course specific siqjport 
participation in peer review of teaching materials 

Area 3: The teaching and learning process 
The instructional component 

ensining student access to resources 
skills in teaching at a distance 
The intemalisation component 

teacher planning for student intemalisation 
variety of student/teacher interactions 

encoxiragement for students to contact other students or 
interested parties as part of their learning environment 
The presentational/assessment compc»ient 

responding to student learning through assignments 

provision of variety/choice in the types of assignment 

requirements 

Area 4: Monitoring and evaluation 

using <^valuativc information within planning phases 

diagnosis of student assignments to seek evaluative information 

formal evaluation 
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For each of the points in these areas a statement provided further elaboration 
- in many instances the elaboration describes process or desired outcome 
thereby providing an 'indicator' of performance. Unfortunately^ a discussion of 
these elaborations is outside of the scope of this paper. However, to give an 
idea of the nature of the case study we provide sample sections of the report 
(Nunan, 1991). The first section reports issues which were discussed during the 
first group session and has been provided to illustrate the ways in which issues 
about quality were seen as problematic. The issues explored were: 

the tension between two forms of 'judging quality'. One approach 
involved the idea of judging what is 'valued-added' for individual 
students. A quality program means that, for the majority of students, 
it identifies the student starting points (entry knowledge/skills) and 
enables the 'adding of value' from this starting point. Judgements 
about quality according to this view stress the learning resulting from 
the program relative to where the students begin from. Outcomes of 
students may be quite unequal even for graduates of the program. 
Another approach is to focus upon student outcomes and insist that a 
certain normative level of achievement occurs across the totality of 
outcomes. An analysis of 'quality' should acknowledge this tension as 
those teaching the program are continually called upon to make 
judgements by applying either or both views of quality, 

the balance of trade-offs between the 'value-added' and normative 
view. These may be set differently for different programs. For 
example, student outcome judged against a normative standard is usual 
for Masters work where external examination of a thesis is a 
requirement. That is, quality is articulated differently for different 
programs, 

the need to place importance upon transactions between (among) 
teacher and leamer(s). Judging quality in education at a distance 
involved the process of valuing transactions for educational worth. 
The basis of the valuing is to be found in the general conceptual 
framework surrounding the idea of education. Education at a distance, 
while distinctive, is not a discrete education genre, (p7) 
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The following section gives a glimpse of the impressions of the investigator 
after the initial interviews: 

First, quality in distance education within a discipline area was seen 
largely in terms of those curriculum and teaching issues over which the 
group had absolute control. The one notable qualification was the role of 
Library in providing 'search and supply' services. Interestingly, the 
Distance Education Centre was seen to carry out a technical role. It was 
seen as a pipeline between the institution and student, which should be 
established as rapidly as possible in order to carry the appropriate course 
materials, communications, assignment traffic, and the like. While such 
services provided by the Distance Education Centre were essential it was 
clear that issues of 'quality' from the viewpoint of academics was not 
about such technical matters, and, instead, involved factors more 
intimately connected with teaching and learning. This can help to 
explain the special importance of library services. They, in effect, were 
often central to students being able to achieve the particular teaching 
and learning strategies which encompass practices that were identified as 
contributing to 'quality' in distance education. Delivery services supplied 
by the Distance Education Centre, on the other hand, were a further step 
removed from values embedded in teaching and learning strategies. 
Because they are rarely initiated by or concerned with the needs of 
individual students, they were 'conduits' of a technical nature. It was 
only when the Distance Education Centre could demonstrate a form of 
partnership or involvement in matters which involve what academics 
view as teaching or learning issues that the Distance Education Centre 
could be seen to have an impact upon quality. For example, one 
academic noted that if the Distance Education Centre had within its 
powers the means to provide 'release-time' for writing course materials 
then it would have an impact upon quality. 

Second, judgements about practices which had an effect upon quality of 
education were seen to be 'holistic'. Further, there were few 
opportunities for the groiq) to formally focus \xpon such judgements. 
Quality in distance education within the area was likely to be formally 
considered only at times of discipline or program, review - and then, 
frequently, this would mean an institution, rather than staff, initiated 
process taking place where evaluative activity would be directed towards 
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accreditation or reaccreditation. Given the pressures surrounding these 
formal processes and the involvement of other interests in course 
evaluation groups, this form of evaluation is often complex and does not 
provide the best opportunities for staff to debate issues which impact 
upon quality. However, for many programs in the institution, this form of 
review is the only forum where such issues are covered. Interestingly, 
while staff agreed that there was indeed a lack of formal focussed 
opportunity to reflect upon issues impacting upon quality, they 
nevertheless held the view that their practices were continually 
appraising 'quality' in some form. As one staff member put it there was 
a 'sort of hidden model in my head, a reference model that I would be 
comparing things with, even though we wouldn't have articulated it and 
put it on paper'. 

Quality considerations, in these terms, are pervasive and continually 
infuse into judgements surrounding practical action. 

Third, the matter of quality in teaching-at-a-distance was seen largely 
as a matter of communicative competence. The key vehicles were 
assignments and journals and, to a lesser extent, teleconferences. 
Frequency of communication was not so much an issue as the timing and 
the type of response. The aim was to encourage interactions between the 
individual students and staff. Interestingly, there was tension and 
uncertainty surrounding the matter of fostering quality interactions with 
students; tensions, because there seemed to be insufficient time available 
to effect an ideal form of individualised response, - uncertainty, caused 
by the often limited information available about the learning situation - 
an absence of any agreed ^standard' which might be applied to judge 
whether a particular response was indeed adequate. Many staff judged 
this area to be the most difficult and time consuming. 

Fourth, staff analysed the characteristics of distance education as ones 
which flowed from the special Mi^uts* to the system. Distance education 
was largely a response, in teaching and learning terms, to these inputs. 
Outcomes were, by definition, those of the course and were absolute in 
the sense that they were ideally the same for both the internal and 
external students studying the course. 
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The inputs that academic staff mentioned in the discussions ranged from 
their knowledge of the learning environments of students to their own 
workloads. For example, it was argued that because external students do 
not have access to libraries a compensatory emphasis should be placed on 
the provision of sufficient resources to enable the student to cope with 
the course; because external study involves negotiating formal 
structures within course materials, staff should be sensitive to any 
intimidation and stress generated by such structures and establish 
processes which lessen such effects; because staff do not receive formal 
workload recognition for preparing external course materials the 
products are of a lesser quality than might be achieved were workload 
recognition given. 

Improving quality in distance education was therefore a function of the 
institution and staff recognising the situation and providing resources to 
standards which would achieve satisfactory processes and outcomes. For 
example, to overcome the difficulties of establishing 'dialogue' between 
students and staff more teaching time would be needed so that responses 
to student assignments could be a vehicle for individualised teaching. 
(pl8) 



This final section from the report again illustrates the complexities of case 
study approaches when analysing something as problematic as quality. The 
group raised conceptual questions such as: 

Is it possible to evaluate the rich variety of decisions about teaching and 
learning through indicators and if indicators of some kind could be used, 
how would the way in which they were organised or groiq>ed reflect 
philosophical positions about pedagogy in education at a distance? The 
group was not happy with the term indicator as it represented for them a 
causal connection between beliaviours. They believed that, at a 
management level when dealing with groups of students, there may be 
some justification in describing and using indicators to account for 
practice, but, at an individual student to teacher level, indicators could 
not describe or be the basis of evaluating teaching or learning. This 
dissatisfaction with the term resulted in the use of the word •elaboration' 
when dealing with aspects within particular areas. The second matter of 
grouping 'aspects' to reflect philosophical positions was an important 
isff.ie with some staff, however, for others it was perhaps more important 
to consider aspects and their elaboration rather than the structure of 
areas* 
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It had been ex{>ected that a classification similar to that used in 
evaluation studies might have been favoured (ie context, input, process, 
and output) as a way of organising teaching and learning activities. 
However, an initial draft of possible indicators organised in this way was 
rejected, (pi 9) 

In summing up, the report noted that: 

staff involved in the study saw the quality of the interaction between 
individual students and the teacher as the key factor in a quality 
program. Theirs was a learner centred view of the educational process. 
They could always illustrate their views by reference to a particular 
student in a particular circumstance discussing the dilemmas arising from 
the teaching and learning decisions which represented their diealings with 
that student. This is indeed interesting as one could speculate that this 
'mind-set' may have developed by having to deal primarily with external 
students over a considerable period. Much of the discussion started from 
a framework about individual learning rather than group based teaching, 
a situation which often requires considerable re-orientation of values and 
i>ttitudes by lecturing staff in a Univereity setting. 

A further point of interest was that the group placed little emphasis upon 
the matter of designing course materials. Instead there was an 
acceptance that it was preferable to review whatever might be produced 
by staff. That is, approaches to preparing course materials were largely 
intuitive. Deciding the form of the teaching dialogue within the 
materials could not be divorced from the substantive content of the 
course. 

While the group was familiar with many curriculum design issues there 
was little interest in materials design - this was not suprising as the 
group placed more emphasis upon an 'intemalisation' process to ensure 
that materials 'came to life' and assumed meaning for their students. 
Design for such unique responses by students during this intemalisation 
represented, in part, a contradiction in terms. (p20) 

We have iii^. Jed considerable detail of this one case study in order to 
illustrate the complex issues arising from judgements about quality in distance 
education. 
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Nomination of Indicators, Study 2: Report and summary of worlc undertaken 
The staff involved in nominating indicators of quality in distance education 
provide post graduate studies in the area of science and mathematics 
education. The student group is mature age and all have completed an 
undergraduate course, have work experience and are currently employed in the 
field in which they are studying. 

The nominated indicators covered quality of intake (indicators of the student 
likely to succeed), quality of study materials and staff attitudes towards 
distance teaching and learning. The indicators and elaborative statements are 
as follows: 



Indicator 

Indicators of the student 
likely to succeed 

already holding a degree 

mature aged 
. employed in the field of 

study being pursued 
« possessing work experience 

Indicates in stud|y material 

• flexible in encouraging 
work related projects 



well provided with selected 
extracted readings 



ElabOTatim 
(where necessary) 



study guides prepared on 
sound instructional design 
principles 



study inclusion of 'generic 
tasks* so that e^^riences can 
be derived and applied to 
practical work environment 

all subjects contain photocopied 
journal articles and/or chapters 
from textbooks that are 
integral to the assignments for 
each -nr^odule. There should be 
enot literature provided to enable 
stiiOi^nts to write a scholarly 
assignment. 



Indicators in staff attitudes 



flexible and understanding 
on deadlines 

thorough marking of assignments 



shoi ; turn around time for 
assignments 



availability to students 



assignment marking is an important 
means of communication and should 
involve written commentary 
documenting strengths and weaknesses 
assignment turn around time within 10 
days 

variety of contact mechanisms and 
encouragement to contact as the need 
arises 
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These indicators represented key elements in determining the quality of 
education at a distance. Emphasis upon interactivity (thorough marking, short 
turn around, instructional design), and resources and support for external 
students, reflect the concerns expressed in the first case study. 

Case study 3: Report and summary of work undertaken 

The staff taking part in this study are mostly from the School of Social Science 
and Welfare Studies involved in teaching in the Graduate Diploma in 
Counselling. One of the participants, however, is from the School of Teacher 
Education because the Graduate Diploma has two strands; a general strand and 
a school counselling strand. 

The Graduate Diploma has been offered for a number of years. The course is 
offered by distance education only and the enrolment numbers are divided 
equally between the general and the school counselling strands. One of the 
attractions of the course is that it has Australian Psychological Society (APS) 
accreditation and counts as a fourth year study of psychology. The quality of 
students enrolled in the course is very high as it is one of the few fourth year 
accredited courses offered by distance education. 

The requirements of the two strands differ as follows. The Education 
Department in NSW funds the places for the school counselling strand and 
offers the support of the department in time off work to attend the residential 
components of the course; and offers placement of students and skilled 
supervisors for the practical components of the course. The students in the 
general strand are required to make their own arrangements about the 
residential components and their placement and have to find a suitably 
qualified supervisor. These are only superficial differences. The central 
requirements of the course are in every other respect identical (Linfoot and 
Todoroski, 1991). 

The course is essentially a skills based course and is a challenge to offer by 
distance education. One of the participants summed up this challenge in 
relation to maximising the opportunities at residential components of the 
course. 

It is a very skills based course in many ways as opposed to the more 
traditional theoretically based, I mean there is a mixture of both, and 
(so) ... the najor quality issue in making sure that those residential 
times are used to the maximum potential; are used for that practical 
aspect of counselling. 
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While this challenge remains, the review of accreditation by the APS in 1990 - 
1991 has meant that more emphasis has been placed on academic research and 
less on counselling skills. This has added two strains on the teaching of the 
course revealed by the comments made. The two difficulties are: the first is 
the tension engendered by the needs of the students to obtain counselling skills 
for vocational reasons; the second is the difficulty in teaching research at 
postgraduate level when the institution is just beginning to review the 
adequacy of its resources to cope with research courses (for example, the 
adequacy of library resources) (Relf, 1991). 

Staff 

The staff who lecture in the Graduate Diploma are all experienced lecturers 
and all but one of the participants has had extensive experience in teaching by 
distance education* Most have been teaching in the Graduate Diploma for a 
number of years and are also teaching in the new Honors course offered by the 
University by distance education and internally. 

The comments made reflect the context of the course and the experience of 
the participants. While most of the comments are confined to the Graduate 
Diploma course* comments on the library resources for research for example 
are applicable to the Honors course as well. Some of the examples apply more 
directly to the Honors course. Comments about the design of courses for 
distance education mode as opposed to the lecture mode are mainly applicable 
to the undergraduate experience of the participants. 

The areas of concern that were raised in the interviews can be categorised as: 

course design 
thinking skills 
assessment 

writing distance education notes, the task per se and time 
residential schools 
institutional support, and 
commtmication with students. 

Methodology 

Participants were approached and asked if they would take part in the survey. 
At that stage the Project and the open ended nature of the interviews were 
briefly described. Individual interviews were arranged and the idea of a group 
session was raised at the initial contact - although, because of the pressures of 
Q the end of the semester and the course reviews resulting from the 

ERIC amalgamation of Charies Sturt University, the idea was only weakly supported. 
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The interviews started with a description of the project and the open ended 
nature of the interview was restated. Participants began the interview 
elaborating on issues of quality as perceived by thetn. 

Participants were also provided with a list of statements about quality in 
education which were used as prompts when the interviews faded. In the early 
stages of the intervi-^w the approach was to ask for greater elaboration and 
clarification. Later in the interview the approach was to refer to the sheet of 
definitions and descriptions of quality or refer to pertinent comments that had 
been made in earlier interviews. 

The taped interviews were transcribed and then summarised and categorised in 
this report. 

Summary of perceptions 



Area 1: Course design 
Aspect 

Integration of all aspects 
of DE package 



Structure of study 
notes 



Elaboration 

The quality of distance education 
was not defined simply by the material 
sent to students but the whole 
educational experience. Thus aspects 
of quality essentially entail the links 
between the vv'ritten material, the 
residential school, interaction with 
lecturer and other students and support 
services such as the library or 
computer centre. 

The instructional technology on the 
presentation of teaching materials was 
important. Breaking text up into 
easily identified components - 
objectives; self motivational 
procedures; inclusion of other media - 
are some of the factors involved^ And 
the basic 3 book format (Information 
book. Study Guid«; and Resource Book) 
was particularly significant to one 
participant. It was generally accepted 
that the course design provide notes 
that are a compendium to other 
resources, and that the notes be fairly 
structured. 
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DE course equivalent 
to face-to-face course 



Content relevant 



Content be up to date 



Better prepared than internal 
courses and underutilised mode 
of study 



Course design around the 
residential components 



Area 2: Thinking skills 
Aspect 

Challenge students 
to extend themselves 



The objectives and outcomes 
equivalent for courses offered 
internally and by distance education. 
Same objectives and outcomes though 
a different course design to achieve 
the objectives and outcomes by DE. 

The courses need to be relevant to the 
vocational needs of the students. Part 
of this is that the materials should 
maintain appropriate to standards 
though be aware of the time 
constraints that the students are 
working within. 

The 'content has to be seen as being up 
to date* becaiise the education 
students frequently compare the 
course to the inservices and with other 
institutions at inservice training. 

*I think distance education is almost 
superior to the normal lecture 
procedures ... probably (because) much 
more preparation and much more 
thought goes into the material.* Much 
more emphasis is placed on working 
out the objectives and meeting those 
objectives in the notes than happens in 
internal teaching. 

Because of the counselling skills 
component of the course the 
residential periods are important for 
that development. Thus the course is 
designed so that there is no doubling up 
with progress through the course at 
home and at residential school: *I 
certainly try to make sure that what 
goes on in the residential school are 
things that cannot be done at home.* 



Elaboration 

Lecturers should stimulate students 
to greater understanding of the 
material than the students realised. 
But this has to be done while being 
sensitive to the students* other 
commitments. 
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Link new learning to 
students level and to their 
experience 

Develop critical thinking 



Area 3: Assessment 
Aspect 

Progressive assessment 



Feedback 

Area 4: Writing 
Aspect 

Time available for writing 



Commitment to the course 



In teaching lecturers need to challenge 
their students to learn more efficiently 
and effectively* 

Part of the linkages that are one aspect 
of quality of learning, the students 
actual experience is one link that 
needs to be made* 

All participants considered the 
development of critical thinking as a 
major aspect of quality of a DE 
course. One considered it to be the 
most important factor* 



Elaboration 

Most participants considered that 
students were served better by 
progressive assessment throughout the 
semester. This was one aspect of 
quality as students tended to work in 
spurts. By progressive assessment it 
helped students attain the goals of the 
course. Some considered it important 
to integrate the assignments with the 
material even to stating the topics 
that an assignment related to. This 
relates to the course design raised by 
some participants. 

Feedback was very important as it 
provided a real opportunity for 
individualised tuition in DE. 



ElaboiatifiQ 

Most were sensitive to this and were 
concerned that there was not enough 
time allocation for writing. This was 
seen as an institutional problem as the 
institution did not allocate time for 
writing. 

Some saw this problem from a broader 
perspective looking at staffing levels. 
That because there was not adequate 
staffing levels* there was not enough 
time for the myriad of jobs within the 
school to be done and a time allocation 
in itself would not be sufficient. 

One considered that the increasing use 
of contract staff potentially led to a 
decrease in the quality of the course. 
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Skills of writing for DE 



Area 5: Residential school 

Aspect 

Integration 

Interaction 

Area 6: Institutional siqsport 



Library services 



Different medium 



Area 7: Communication 

Communication as student 
support 



Good skills of writing instructional 
materials were a contributing factor 
for quality. The ability to link ideas 
and paragraphs is an important skill 
that needs to be addressed by 
institutional support for writers- The 
conversational writing style was 
adopted by some while some 
considered the need to provide 
motivation in the written material the 
significant skill* 



Ela 



All considered the integration of the 
residential components into the whole 
teaching package as an essential 
component of quality. 

The interactions of students and staff 
as well as with other students were 
also essential components of quality. 



Elaboratipn 

Library services are important for 
Siaff in keeping up to date with their 
specialisation and for students in the 
usual way and for developing research 
skills. These two factors were 
mentioned by most of the participants. 

Residential schools, teleconferencing, 
audio tapes and the vnitten materials 
were all important in development of 
high quality courses. Mast would like 
assistance and the encouragement for 
using video tapes particularly for 
demonstrations of counselling 
techniques. 



Elaboration 

All communication with students 
was important. Starting with the 
design and language of the notes, 
assignment feedback, return of 
telephone calls and contact during 
residential school were all part of the 
integrated theme that most 
participants emphasised. 



ERLC 



IK) 



-108- 



Discussion 

The project investigators had hoped for a number of studies which would cover 
a range of discipline areas, a range of level of awards, and, a range of delivery 
techniques. While the studies presented by the project are useful and provide 
'indicators' of quality in distance teaching and learning it is clear that there is 
need for further extensive work in recording practitioner theories and 
practices in the area of quality improvement of distance teaching and learning. 

The studies point to the importance of the following factors: 

the involvement of expertise/staff development at the planning or 
review stage of course development or i edevelopment 
the need to ensure adequate resources are available prior to the 
teachinf of an external course to enable course preparation, writing, 
editorial consultation, media support etc. 

the need to place the means of interactivity with the student at the 

centre of distance teaching and learning. This includes the use of 

residential schools as well as mediated conferencing 

the need to ensure that external students receive adequate and 

equitable levels of academic and administrative support during their 

studies. 

The project investigators feel that it is useful to note the fact that these 
exercises with this stakeholder were viewed as a useful form of staff 
development activity by all participants as they formed a type of 'discipline 
review'. Also, the carrying out of this part of the project was a complex 
activity involving a range of evaluation and ethical issues regarding the 
purposes of the study and possible uses of the results of the study. Not 
unexpectedly, the timing of the activity, the availability of on-site 
investigators, the establishing of support with staff in the discipline area, and 
the support for such activity by the Schools/Faculty of the participating 
institution were all factors which had an impact upon both the practicability of 
conducting a review and the extensiveness of the review itself. 
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Appendix 3 

INSTITUTIONAL CLIENTS OF DISTANCE EDUCATION CENTRES 
THEIR CONCERNS FOR THE QUALITY OF DISTANCE EDUCATION 

Introduction 

This appendix is concerned with the term of reference of the Project which 
considers the needs of potential client institutions by seeking input on their 
views about how Distance Education Centres might assist them in achieving 
excellence in distance education. The investigators sought advice on the 
methodology of obtaining 'inputs from different clients' as it was felt that 
•client satisfaction' in relation to the quality of services and products of 
Distance Education Centres may be uniquely determined by the needs and 
nature of specific contractual relationships developed between a client and a 
particular Distance Education Centre. Institutional clients are not a 
homogeneous group - the needs, expectations, and educational background of 
corporate or private clients were different from clients within the higher 
education system - also, it was clear that there were wide variations between 
higher education clients of expertise, experience, and infrastructure to support 
distance education. 

After advice from various sources the investigators decided to seek: 

initial views about the involvement of higher education institutions not 
designated as DECs through the meetings of their operational 'heads'. A 
letter seeking advice on the most useful form of input was circulated to 
this group through Associate Professor Bruce Scriven of Queensland 
University of Technology and members of the groyxp were invited to 
respond during a teleconference session. A summary of responses was 
made available to the Project. 

expert advice from Professor Faith Trent, a member of NDEC. The 
Flinders University of South Australia provides nursing and education 
awards externally and Professor Trent has responsibilities for matters 
related to Flinders University as a 'client' of the Distance Education 
Centre of the University of South Australia. The interview with 
Professor Trent focused iqx>n policy issues related to quality 
improvement of distance teaching and learning through the services 
provided by Distance Education Centres. 

input from three private 'clients' *-* Engineering Education Australia, 
Telecom and the Australian Taxation office. An initial response from 
Engineering Education Australia suggested a way in which the responses 
from Telecom and the Australian Taxation Office might be organised. 
Consequently these institutions were invited to comment upon: 

4-17 
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materials presentation - content, structure and design, physical 
presentation 

teaching/delivery - assignments and examinations (appropriateness, 
weighting, relationship with course objectives), interactive 
elements/communication technologies, tutor and support mechanisms 
(availability of counselling, availability of resources for learning etc) 
administrative support (dispatch pre-enrolment and post enrolment 
information, interstudent liaison processes). 

Higher education institutions as clients 

The advice from representatives of non-DEC higher education institutions was 
ambivalent with regard to the need to consider the DEC/non-DEC relationship 
as a part of the study of quality of distance education. There was some 
support for the notion that each client would inevitably 'measure' the quality 
of services provided in terms of expectations established within a contractual 
relationship. Indeed, it would be up to the client to spell out sufficient detail 
within the contract in relation to its expectations so that it was clear that at 
least a minimum acceptable standard was met by both parties. In clarifjang 
such matters of detail it was thought that the indicators nominated in Chapter 
5 would provide a useful checklist of factors which might be applied to their 
particular circumstances. 

The interview with Professor Trent focused upon wider quality 
management/improvement issues and the possible future roles of Distance 
Education Centres. In the medium term (5 years) it was thought that non-DEC 
higher education institutions would be likely to need: 

advice/assistance with curriculum and materials design for external 
teaching and learning ma terials 
editorial/graphic design services 

specialist design advice and some production support for non-print 
components of learning packages 

advice and services which would enable a student siqjport dimension for 
external students to complement existing administrative systems, 
the use of specialist communication technologies/software etc available 
through the Distance Education Centre for students of the non-DEC 
institution. The use of these facilities is especially important in raising 
the quality of distance education and implementing a ^culture* which 
acknowledges and accommaiates the needs of external students. 
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Institutions designated Distance Education Centres should act as key centres in 
at least two ways» First they should trial and introduce new technologies 
which facilitate teaching and learning at a distance and disseminate these 
developments through consultancies and staff development within other higher 
education institutions. Second, Distance Education Centres should provide a 
expertise to other higher education institutions which would enable them to 
initiate staff development programs for the improvement of distance teaching 
and learning* Non-DEC institutions should be aware of the range and level of 
expertise available and institutions should contract DECs to deliver such 
consultancy and staff development services which should be available as a 
normal part of non--DEC institutional e)q>enditures on the improvement of 
teaching and learning. 

Other clients: Engineering Educaticm Australia Pty Ltd 
In the response provided, Engineering Education Australia indicated that, as a 
consumer, it was reflecting the interests of over 100,000 professional engineers 
- however, they believed that these needs were not so different from the needs 
of other distance learners thereby noting that their response would have a 
broad application. 

Quality, in the view of this client, needs to be defined 'not only in terms of 
high standards of design of learning materials, dynamic interactive teaching 
methods and efficient/sensitive student support services, but in terms of 
Access, Flexibility and Relevance*. The response continues noting that 'the 
key word is flexibility, since this quality or characteristic is essential if access 
and relevance are to figure significantly in the kind of programs that are 
offered to and accepted by the engineering profession or other similar 
professional groiq)'. 
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'Flexibility* as a prime characteristic of a quality program would entail, in 
their view: 

providing more opportimities for professionals to relate the material to 
their respective workplace contexts, for example through the provision of 
relevant case studies and setting project-oriented assignments 
developing imits in a modular form so that professionals have choices in 
the size of their learning ^bites* ... 

improving and e}q>anding the use of communication technologies in 
distance education, especially to provide for more effective peer groi^ 
interaction providing teaching/learning e)q)eriences in a variety of 
settings and delivery modes, eg on-site tutorials or workshops, field 
studies, teleconferencing, computer conferencing etc. 

1^9 
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providing effective pre-enrolmcnt information about objectives, content, 
structure and delivery of distance education courses, essential course 
requirements and special features with implications for student decisions, 
and most importantly, the target group or individual for whom the course 
is intended. 

In providing guidelines to quality the response provided information in the 
areas of learning materials (content, structure and design, physical 
presentation), teaching/interactive elements, and administrative support 
(dispatch arrangements, student support). As a part of the pre-^nrolment 
information available to students the response indicated that the following 
information should be routinely available in standardised format to enquirers - 
number and name of subject, adviser/coordinator, semester offered, 
prerequisites, subject outline or description, learning materials provided, 
prescribed texts, essential equipment required, attendance requirements, 
special teaching methods (interactive elements), assessment. 

It is significant that Engineering Education Australia commissioned a report, 
Delivery and comnmnications technologies in the provision of professional 
continuing education for engineers (Guiton and Atkinson, 1991) to advise upon 
possible uses of technologies. A recommendation from this report notes that: 

EEA inform imiversities that the extent of use of interactive 
com mimicat ions technologies is a significant component in the covirse 
quality assessment criteria to be used on the proposals to EEA. 

The report notes that ^firstly, in general a course with good use of interactive 
commimications is very likely to be a better quality course. This may be at a 
simple level, such as high standards in individualised telephone tuition or at 
more complex levels such as audio teleconferencing or video 
teleconferencing*. In line with these sentiments a further recommendation 
states that 'EEA encourage in its requests for proposals the systematic 
provision of access to interactive communication technologies, including 
computer conferencing, electronic mail and teleconferencing. Ways of 
facilitating both student and EEA access to these technologies should be 
carefully assessed for inclusion in courses proposed for EEA approval*. 
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Thus it is clear that, from this client perspective, technologies which allow 
conferencing (ie where receivers and originators have similar media 
capabilities) either in real time or 'asynchronously' are seen to provide an 
added dimension to the quality of the course and its delivery. 
Recommendation 18 (see Chapter 6) is directed towards starting the process 
enabling institutions to strengthen the use of computer mediated 
communications within distance teaching and learning. 

Other clients: Australian Taxation Office 

The Australian Taxation Office is involved with the delivery of education at a 
distance through the University of New South Wales and the University College 
of Central Queensland. 

The following listmg provides this client's perspective upon factors influencing 
the quality of distance education. 

Materials Presentation 
Conten t 

The material content must be academically soxind; 

The content must be validated by independent experts, preferably from 
outside the DEC system and with relevant current exposure to industry; 
The content must be kept up to date. 

Structure and design 

The materials must be educationally soimd, with predetermined 
objectives set out for students to read and understand; 
The material must be easily referenced; 

Uniform minimum standards in terms of quality of print and presentation 

should be developed. These standards should cover such things as: 

. the resolution of the printing, 

. ease of incorporation of amendments, 

. regular review and iqxlating of materials. 

The structure and design of the materials should be consistent from 
subject to subject throughout the program. 

Physical presentation 

The materials must have a professional appearance, e.g. they should be 
properly bound, not just stapled; 

There needs to be an appropriate balance between professional 
appearance and cost, with neither one dominating; 
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Presentation has to be such that it is not off-putting to the student, e-g. 
. pages overcTOwded with type that is difficult to read; 
. the size of the material package is so big as to be daunting for the 
student; 

- The material must not look out of date. 

Teaching/delivery 

Assienmen ts and examinations 

Assessment weightings should not be dissimilar from on-campus courses; 
Assignments and examinations should assess whether the course 
objectives in terms of skills and knowledge have been met. 

Interactiv e elements/communication technolog ies 

There should be minimum standards for the time period for answering 
student inquiries and providing assignment feedback; 
There should be appropriate use of available technology. Its use should 
be judged in terms of the value added to the student learning experience 
and cost saving, if relevant; 

Communication technologies should only be used where they are easy for 
students to learn. 

Tutor sup port mechanisms 

Use of videos to introduce the lecturer and the subject to the students; 
Students should be encouraged to be self-supportive, for example, the use 
of learning groups amongst the students should be promoted and actively 
encouraged. The University should take an active role in the 
establishment of the groups; 

There should be easy access to student counselling services. The 
university should take a pro-active role in monitoring the progress of 
students, for example, they should follow up on late assignments to 
forestall student dropout; 

The university should initiate telephone communication with all students 
early in the course. Ideally this should be done by both the lecturer and 
the student counsellor. 
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Administrative support 

Materials should be dispatched early to students; 

The university should make use of Newsletters and bulletins both before 
and during semester to keep students informed; 

There should be easy access to critical information via a student hot-line 
or similar service. 

Other clients: TELECOM 

The response from TELECOM was based on initial experience derived from 
Telecom Training Services role in introducing studies in an Associate Diploma 
of Adult Education (Training), Maximising retention rates was linked to the 
quality of a program and importance would be attached to: 

enhancing student motivation particularly through mechanisms to 
recognise prior learning within the assessment of the course by providing 
ci-edit for prior learning, 

the links between competency profiles and career development, 

a participant si^port framework through 'in-house' tutors. 

flexibility in meeting student needs through 'extended completion 

periods' and individually negotiated contracts. 

modelling good delivery through appropriate use of technology - e,g, 
bulletin boards for E mail etc. Interactivity is essential and the delivery 
should exhibit a clear break from 'correspondence models* of distance 
education. 

Discussion 

Views about the qxiality of distance education from various clients depend upon 
their needs in relation to course design, delivery, requirements of professional 
associations etc. Interestingly, corporate clients are using the mechanism of a 
contract or proposal to make specifications on factors that they believe will 
influence the quality of the distance education for which they are contracting 
services. In addition some clients ask that a 'quality manual' accompany each 
subject or xmit for which materials and distance teaching/open learning 
arrsingements apply. 
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APPENDDC 5 

ANNOTATIONS OF SELECTED WORKS 

The following annotations are the author synopsis or have been constructed 
from key sections of the authors text. The annotations are intended to provide 
a guide for those who wish to follow up on matters of detail in areas related to 
the quality of education* The annotations provided are from a selection of 
works within the bibliography. 

Acherman, Hans A. (1990) Qmlity Assessment by Peer Review; A New area for 
University Cooperation, Higher Education Management, 2, 2, 179-'192. 
This article reports on the progress of quality assessment and the new 
relationship it is forging between the national government and the universities 
in the Netherlands. It also describes a project on quality assessment started by 
the Association of Cooperating Universities in the Netherlands in which the 
quality of twenty six university programs was assessed on a nation wide scale. 
This assessment was undertaken by four visiting committees of independent 
experts. The author concludes that although aiming in the first place at the 
improvement of educational quality, this project is also a major step in the 
direction of more self regulation on a departmental and institutional level and 
of less interference by the government. As far as circumstances in E\iropean 
countries are comparable, universities could leem from each other. This 
article suggests joining forces on the practice of quality assessment, the 
European year 1992 approaching rapidly*. 

Altdn, M. (19m National Quality Indicators: A World View, Studies in 
Educational Evaluation 14, 1, 11''24. 

•This paper presents an overview of the issue of national quality indicators 
based partially on a synthesis of the informed judgements of a number of 
respondents from throughout the world. • It presents an analysis of the quality 
indicator movement and its focus on 'the development of new measures, 
procedures for standard setting, better procedures for data collection, storage 
and analysis'. The author categorises indicators as inputs and outcomes. 
Inputs are defined as schooling resources and processes. Outcomes are defined 
as the acquisition of knowledge. He then goes on to survey the means by which 
these indicators arc measured and in particular points out the ^apparent 
conflict surrounding the use of the test performance of individual students as 
the basis for judging quality of schools*. His discussion of the role of quality 
indicators also points out the tenuous relationship between evaluation 
information and direct administrative action. 
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Ashenden, Dean. (1988) Indicators, Information and Edtication: A discussion of 
possibilities and problems, CDC, Canberra. 

•This paper considers both possibilities and desirabilities in the use of 
indicators and, more broadly, in the use of information about education ... Its 
general conclusions are that it is ... possible to collect and use valid and 
reliable information about almost any aspect of the educational process.' It 
discusses the benefits and dangers of this process and points out the important 
role to be played by schools and systems. 

Baldwin, Hon P. (1991) Higher Education: Quality and Diversity in the 1990s 
AG PS, Canberra. 

The purpose of this paper is to 'take stock of the general impact of the White 
Paper policies, to respond to new issues that have emerged, and to chart new 
directions for the future'. It draws from the views of higher education interest 
groups and also from a number of recent reports in the area. 

'The statement sets directions for the development of Australia's higher 
education system through the 1990s as a foundation for our social, economic 
and cultural development in the twenty first century. These directions build 
upon the achievements of growth with equity in participation that have been 
realised through the White Paper initiatives. They give priority to quality in 
diversity as the jsystem responds to emerging challenges ... The focus of the 
new policy directions is the development of a comprehensive set of measures 
to further enhance the quality of higher education teaching and research.' 

Boll, R. and Halwachi, J. (1987) Performance Indicators in Higher Education, 
Higher Education, 16, 4, 393^5. (author synopsis) 

'Interest has been increasing in many countries in assessing performance of 
higher education institutions (and also departments within institutions). Two 
important reports in the U K in recent years have strongly advocated the use 
of performance indicators for this pur^^ose. This paper argues that such 
indicators cannot be used in a meaningful way without a clear view of 
institutional goals. Problems of deriving such goal systems are discussed and a 
critical review of work done in this field is presented. Methodological 
problems associated with devising and using effective and useful performance 
indicators are described and research undertaken in this field outlined. Finally 
the authors outline the conditions under which they believe performance 
indicators may be used to give valid insights into performance of institutions 
(or departments)'. 
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Barache, J. (1988) Total Quality Policy in Distance Education as a General 
Process of Innovation, Epistolo Didaktika, 1, 11-30: 

This paper discusses the definition of quality and how it might be applied to 
distance education* The paper defines the concept 'total quality' as applied to 
distance education and identifies the seven service components that contribute 
to total quality as accurate information, ethical guidance, enrolment 
formalities, the self-learning process, continuous relationship between the 
institution and the student, psychological stimulation from the institution and 
administrative system to help the student to carry on and not drop out, and 
assistance at the end of the course in practical decisions connected with the 
conclusion of the learning period, most often in finding a job. 

Beswick, D.G. (1988) Prospects for the 1990s: a new phase of development in 
Australian Higher Education: Higher Education Quarterly, 42, 2, 162-178. 
(author synopsis) 

'Higher education in Australia is seen as having reached the conclusion of one 
phase of development characterised by a high degree of central planning and to 
be entering a new phase in which it is argued that a radically different 
orientation is required. Recovery of at least shared responsibility by the states 
and the development of private initiatives should help to promote more 
creative management through diversification of fimding sources and lines of 
accountability. Pressures for change follow demographic and social changes in 
the Australian population which are likely to favour increased participation in 
higher education for both personal and national economic purposes'. 

Blake, Cliff and Wilson, Geoff (1991) Distance Education in Australia, Paper 
prepared for the AV CC, roneo, 1-9. 

The paper provides an overview of the characteristics of distance education 
and its role within the higher education system. The policy background to the 
current system Is outlined and futiire issues for the AV CC are identified in 
relation to the establishment of a single Open Learning Institute and the 
proposed National Open Learning Agency. 

Bourke, Paul (1986) Quality Measures in Universities. CTEC, Canberra. 
This study investigated 'the possibility of defining criteria for distinguishing 
between Australian university departments in terms of the quality of their 
performance.' It also investigates the policy uses to which this criteria may be 
put. An example of the application of such criteria is provided through the 
University Grants Committee's recent practice in the United Kingdom. The 
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experience of the United States in applying performance measures is also 
discussed. Further discussion is made on the issue of quality research relating 
to Australia. The report concludes with the assertion that Australian 
universities need to develop a more energetic culture of evaluation' if they 
are to engage in negotiations with external authorities on terms which the 
'universities have themselves identified'. 

Church, CM. (1988) The Qualities of Validation. Studies in Higher Education, 
13, 1, 27-49. (author synopsis) 

Demands for enhanced quality and quality control in higher education, such as 
those contained in the 1987 White Paper, often involve increased stress on 
validation. This emphasis on a peculiarly British system of peer review of 
courses suggests that an examination of some of the qualities of validation may 
be helpful in ensuring that it continues to achieve the ends expected of it. 

The structure of validation generally requires action at course (or department), 
faculty and institutional levels. Validation can operate in at least three 
modes: initial approval of new courses, regular monitoring of performance and 
intermittent but more intensive inspections of course organisation, and 
achievement. In all three modes the basic structures function in a way that 
uses specific inputs processes and outputs. Among the inputs, are admissions, 
resources, staffing and curricula. The processes involve such feedback 
mechanisms as continuous assessment together with staff development.' 
Training in validation skills is but rarely provided. Outputs are reviewed 
through final examinations, the external examiner system and outside 
visitations. 

The question which arises is whether this system is an adequate form of quality 
control. The conclusion is that, while it has certain methodological 
weaknesses, validation is a legitimate and defensible tool. But, if it is to 
succeed in new circumstances, more thought about its qualities and the means 
of developing them is necessary. 

Conmonvfealth of Learning. (1990) Quality through distance. Condeam, 1, 2, 
1-4. 

This paper discusses the possible contribution that distance education can make 
to the general improvement of education. It compares the level of quality 
provided by internal and external courses and concludes that it is the quality of 
the teaching materials that determines the quality of distance education 
courses. It also discusses the contribution that distance education can make to 
management techniques through the integration of work and study. Use of 
distance teaching materials is also seen to increase access and equity. 

133 
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Cowan, J. (1990) Report of the Quality Tuition Working Oroup; Optimum 
Academic Support for All Students, Paper for The British Open University, 
roneo, 1^26. 

The report uses two conceptions of quality to conceptualise quality tuition in 
the context of the Open University (UK) and covers quality tuition as an 
intrinsic part of course production and maintenance and within the broad 
context of academic support. The latter section of the report focusses upon 
the supports which underpin the delivery of quality tuition and includes a 
student and tutor charter and recommendations for staff development. 



Cuthbert, R. (1988) Quality and Management in Higher Education, Studies in 
Higher Education, 13 1, 59-68. (author synopsis) 

*This paper examines the relationship between quality and management in 
higher education. It considers the nature of management, the meanings 
ascribed to management and related concepts such as leadership, and the 
appropriateness of these concepts in the context of higher education. These 
preliminary considerations provide a basis for considering the connections that 
may exist between management performance and institutional performance. In 
addressing these issues* the paper aims to confront such questions as: what 
part does management play in ensuring high quality institutional performance? 
to what extent can judgements about management performance be separated 
from judgements about institutional performance? and so on. This necessitates 
consideration of how these kinds of performance can be judged, and the paper 
offers frameworks for analysing judgements of institutional and managerial 
performance'. 

Cuttance, Peter. (1990) Performance Indicators and the Management of Quality 
in Education, Keynote Address prepared for the Third National Conference on 
Indicators in Education, Canberra, Deceni>er. Performance indicator Task 
Group, Scottish Education Dept. (author synopsis) 

'This paper deals with a range of issues which impact upon the use of 
performance indicators in managing the quality of education. It commences by 
discussing measurement issues and the purposes for having performance 
indicators. These purposes are then linked to recent change in the 
management of public sector organizations ... Issues of review and evaluation 
in public sector education systems are discussed in the light of a convergence 
of theoretical frameworks from two quite different perspectives: the 
assurance of quality, and organisational perspectives of education systems as 
multi level systems. The role of monitoring is linked directly to the emergence 
of an emphasis on auditing. Audit, review, and evaluation at system level are 
discussed in the context of the assessment of effectiveness, efficiency, and 
ERJC quality assurance. Organisational development and accountability are 
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discussed in the light of the management of quality within organisational units 
and across the system The issues raised in the paper are discussed in the 
context of the emerging system of quality management and review in the South 
Axistralian Education Department/ 

Cuttance, P. (undated) Performance Indicators for Schooling: A Report to the 
Scottish Education Department, (author synopsis) 

'This report reviews and analyses recent developments in the use of 
performance indicators in school systems. Its purpose is to provide an 
overview of these developments and to provide a context for work currently 
being undertaken HM Inspectorate to assess the suitability and prospects for 
the introduction of performance indicators for schooling in Scotland, 

Section 1 of the report provides a discussion of the issues that have prompted 
the introduction of performance indicators in various national education 
systems. Section 2 consists of a series of chapters which address particular 
issues that provide the necessary framework for performance indicators 
Section 3 consists of a single chapter which reports on the development and use 
of performance indicators for education in three nations: the USA, Australia 
and Britain Section 4 considers performance indicators in the context of the 
improvement of schooling ... Section 5 discusses a set of issues related to 
specific considerations of the Performance Indicator Task Group'. 

Cuttance, P. (1991) Monitoring Ediu^ational Quality Through Performance 
Indicators for School Practice, Paper for the American Educational Research 
Association Anniud Meeting, Chicago, 1991. (author synopsis) 
The first section of this paper *^discusses the management of quality in 
education systems. Indicator systems are discussed in the section that follows 
and the relationship between indicators, development and accountability is 
addressed in the context of a system of education review that has recently 
been established in South Australia. The final sections of the paper discuss the 
development of performance indicators to serve both development and quality 
management purposes. These indicators have been develop>ed as part of a 
broad framework for continuously monitoring and reviewing the quality of the 
education provided for students. Information from the indicators of effective 
school practices and functioning*^ is also discussed. 
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Cutting, B.A. (1991) Total Quality Management (TQM) - Everyone is an 
Evaluator, 1991 National Evaluation Conference, Adelaide, South Australia. 
This paper discusses the use of Total Quality Management and its relationship 
to evaluation as a part of Project Management and Budgetting (PMB). 'This 
paper suggests that there is a significant degree of compatibility and 
commonality between TQM and PMB, and that in the respect of evaluation, the 
link is quite strong'. A definition of TQM is provided and the ties between 
TQM and evaluation are identified. The author then goes on to point out 
benefits of TQM in terms of staff involvement. The paper concludes that Mn a 
practical sense, if not in a pure theoretical sense, there are strong similarities 
between evaluation and TQM. Moreover the application of TQM like principles 
to an organisation has the potential to involve virtually all staff in what are 
essentially evaluative practices - thereby enabling them to contribute 
constantly to effective and efficient resource management - and to make 
evaluation of performance an ongoing concern rather than a one-off 
consideration'. 

Dawkins, J. (1988) Higher education : A policy statement, AGPS, Canberra. 
(author synopsis) 

*This policy statement sets out the Commonwealth Government's strategy for 
the long term development of Australia's higher education system. It marks a 
new era of growth and opportunity for our higher education institutions, with 
potentially significant benefits for all Australians. 

The release of Higher Education: A Policy Discussion Paper in December 1987 
led to a process of intensive community discussion and debate on the roles and 
objectives of the higher education system. Over 600 written responses were 
received from major educational bodies, institutions, professional organisation, 
government authorities, employer groups, unions, academics and ether 
interested individuals. In addition, the Commonwealth undertook an extensive 
round of campus visits and consultations with key organisations. The views 
expressed in written responses and by other means, both formal and informal, 
have been carefully considered in the development of this statement. The 
level of debate and interest generated by the Policy Discussion Paper reflects 
the strong wish of many Australians to share in decisions taken on the future 
development of our higher education system. 
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The Government's strategy for the development of the higher education system 
is part of a wider agenda of reform spanning all elements of the employment, 
education and training portfolio. The Government is committed to growth and 
quality enhancement across the whole of the education and training system - in 
schools, technical and further education and industry training, as well as higher 
education. Details of the Government's objectives in these other areas have 
been annountcd in previous policy statements, notably Skills for Australia 
(September 1987, A Changing Workforce (May 1988) and Strengthening 
Australia's Schools (May 1988)'. 

Farren, R. and Guiton, P. (1987) Standards of Quality in External Studies, 
TAPE External Studies College, Western Australia/Murdoch University, roneo, 
I-I2. 

Standards of quality must be viewed within a context. This paper seeks to 
initiate a dialogue on statements of criteria by which quality in external 
studies may be judged. 'The establishment of standards of performance 
assumes a process or product for which there can be a common yardstick. In 
any educational enterprise this must be the learning achieved by students in 
the system. But if we confine ourselves to measiiring learning outcomes we 
will not arrive at guides to operational effectiveness. 

'A further problem is that of establishing a common frame of reference for the 
contexts in which the learning takes place'. External studies and distance 
education should not be treated as synonymous terms. 'Australian universities 
grant the same awards to all graduates regardless of study mode .«.' Standards 
in external mode should therefore not be treated in isolation. ' ... any attempt 
to establish starjdards of quality for external studies, in isolation, could lead to 
decisions being taken which though appearing to benefit external students, 
could seriously disadvantage both on campus and off campus students'. 

Fenwick, John. (1991) A question of quality: aspects of quality in distance 

learning. Paper presented to the 1991 BiennM Forum of The Australian and 

South Pacific Extermzl Studies Association, (author synopsis) 

'In 1988 the Australian Government White Paper - Higher Education: a policy 

statement was released by the Department of Employment, Education and 

Training* It set down the need for an improvement in the quality of external 

studies. 
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This paper examines the definitions of quality, efficiency and effectiveness. 
The paper puts forward the view that quality distance learning must emphasise 
the needs of the learner. The emphasis currently is on content. That emphasis 
needs to be balanced by attention being paid to the process of learning. 
Quality in distance learning needs to develop enhanced metacognition by 
distance students. 

The paper seeks to apply some of the commercial principles concerning total 
quality management. It examines the emphasis on the needs of the customer 
rather than on the institution. 

A change of focus is needed. Quality distance learning requires quality 
distance teaching*. 

Franklin, J. Performance Indicators in Ediu^ation: The Participatory Approach 
in Wyatt, T. and Ruby, A. (1988) Indicators in Education: Papers from the First 
National Conference, Australian Conference of Directors-General of 
Education, Sydney, 45-53. 

This paper disciisses the subject of performance indicators in education with 
reference to their use in the national science funding policy. 'Science policy, 
like education policy, has made increasing use of quantitative **tools" in recent 
years ... The concerns of science policy ... overlap with those of education 
policy, particularly around the question of funding for higher education 
institution (HEI) research ... The challenge for those receiving funding is to 
improve performance and to demonstrate accountability in exchange for public 
support and in order to compete for fewer resources.* The author points out 
the lack of decision making tools available for making decisions about funding 
because • ... no single group Oi academics can provide an overview of research 
activities in all Australian HEIs. Further ... there are no disinterested peers'. 
The British experience in dealing with these issues is referred to as an example 
and the lessons learnt are discussed. 'These lessons from the UGC experience 
raise a question about the difference between top-down and bottom-up 
approaches to evaluation and performance indicator development.' The author 
concludes that while indicators can only be agreed upon in a context specific 
way the bottom line is th^t governments do not have time to do everything 
in a bottom up fashion*. His solution is to develop a participatory approach and 
this is described in the remainder of his paper. 
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Frazer, M. Quality assurance in Higher Education, Council for National 
Academic Awards, UK, (roneo), 1^24. 

•This paper is structured around three questions. Why is there concern for 
quality? ¥^t is meant by quality? How can quality be assured? Approaches 
to answering these questions are described by reference to higher education in 
some countries* However, no attempt is made to provide a comprehensive 
review of the arrangements for dealing with quality in higher education in 
every country. Reference will be made to key publications by authors from 
various countries. Terminology used in the 1? on quality in higher 

education can be confusing because there is n -zit oiy the meanings 

attached to various terms. It is necessary, tnerei . , start With definitions 
and explanations so that the reader will know how to interpret the author's use 
of some much used terms in this field. 

The final section contains a review of the various types of national agency 
concerned with qiaality in higher education. The case is made for an 
international network of such agencies'. 

Gale, F. (1991) True Quality entails 'vive la differenced The Australian, 
Higher Education Section, September 4, 14. 

This paper discusses the possible negative effects of the push for quality and 
equality in higher education institutions. The author believes that relative 
freedom is required in universities if worthwhile research activities are to be 
undertaken. The application of centralised performance indicators, 
particularly those associated with economic control, will not achieve this. 
Greater equity will also lead to an inefficient uniformity between institutions 
which will, in the end, not meet students' needs. ^Universities are not 
businesses. We cannot measure productivity in terms of unit costs or time 
taken to jbtain degrees. Students are not products. Research is not about 
improving our economic performance*. 

Guri, S. (1987) Quality Control in Distance Learning, Open Learning. 2, 2, 
16-21. 

•Distance teaching universities vary considerably in their characteristics but 
share a common concern in the need to ensure the quality of their products*. In 
this paper the merits and problems of the approach used at Everyman's 
University (EU) in Israel are discussed. The major feature of EU*s course 
production is that its academic work is based within a '... nucleus of internal 
faculty members working together with outside contributors from other major 
Israeli universities'* The remainder of the article describes the effect that this 
O has upon the 'control mechanisms' for course approval and design. In brief 
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these include the evaluation of course proposals by outside experts and subject 
area committees. The course is then developed by a course team. The 
benefits of this approach, the author points out, are that 'a small staff can 
manage efficiently a relatively large scale distance teaching operation by 
tapping the intellectual resources of other universities and using special quality 
control procedures*. 

Hattie, John. (1990) Performance Indicators in Education. Australian Journal 
of Education, 34, 3, 249-276. (author synopsis) 

^Educationalists need to participate in deciding how performance indicators 
will be used, if at all and how they will be combined to optimise the quality 
aspects of teaching, research and service in departments of education. The use 
of performance indicators in the United Kingdom, United States, and Australia 
is noted and a model of performance indicators is developed. The model is 
assessed by applying it to departments of education in universities throughout 
Australia. The departments are ranked on inputs, processes and outputs and 
the implications of various weighting schemes are discussed*. 

Hunter, D. (1988) Review of Ensuring Quality in Open Learning: A Handbook 
for Action, Open Learning Branch, Manpower Services Commission, in Open 
Learning, 3, 3, 61-^2. 

The goal of this book is to enable readers to identify the professional skills 
which producers and deliverers of open learning need to provide quality open 
learning*. The handbook contains five chapters that cover the ^process of open 
learning and the need for quality, a suggested code of practice, and a series of 
action guides*. 

Johnes, Jill and Taylor, Jim. (1990) Performance Indicators in Higher Education 
(UK Universities), The Society for Research into Higher Education and the 
Open University Press, Buckingham, (author synopsis) 

*It is ... the purpose of this book to e^qplore the possibility of constructing 
performance indicators for the university sector. Given the inevitable 
resource constraints, we have focussed attention on a limited range of 
variables which purport to measure the teaching and research outputs of 
universities. Furthermore, we made an early decision to concentrate our 
attention on the university sector as a whole rather than focus on any one part 
of it. We therefore set our sights on comparing the performance of 
institutions, using a range of variables which have been frequently mentioned 
as potentially useful performance indicators by policy makers and university 



140 



- 133 - 

administrators. Initially we concentrated on what may be termed 'teaching 
indicators* namely the undergraduate non completion rate, degree results and 
the first destination of newly qualified graduates- This list was subsequently 
extended to cover unit costs and research performance." 

Johnson, R. (1990) Open Learning: Policy and Practice, A discussion paper 
comrmssioned by the National Board of Employment, Education and Training 
and the Department of Employment, Education and Training. AGPS: 
Canberra.{zuthor synopsis) 

'Australian Society is changing. Open learning is an educational approach 
required to cope with those changes in the provision of education. 

Open learning is a term increasingly used in educational writing. It embraces 
such practices as open admission or more flexible admission requirements to 
educational coin^es; student choice of topics and modes of study student 
choice of timing and manner of assessment; use of communication 
technology; to facilitate choice and learning. It is more of an approach to 
education than a particular technique. 

Elements of open learning have existed in Australia for decades and the 
practices appear to be increasing. Some institutions overseas are established 
entirely on an open philosophy. This paper gives some examples. As Australia 
tries to recruit more and more students into tertiary education, and as it seeks 
increases for its entire workforce, the open approach will be needed because of 
the wide diversity of background and needs amongst those being trained and 
educated. This has implications for: 

* policies on admission to courses and training programs; 

* structure of syllabuses and whole courses; 

* the teaching process, less professorial and more tutorial; 

* use of technology, and cooperation in that use; 

* cooperation between institutions and sectors including the private sector; 

* assessment, accreditation and awards; 

* credit transfer and the concept of a credit bank; 

* attitudes of teaching staff, and their conditions of work; and 

* funding and administrative structures in education. 

This paper explores these issues briefly and canvasses the potential advantages 
and risks of the open approach. It concludes that it is probably in the long run 
irresistible in a democratic society; its introduction can be facilitated by 
proper understanding and intelligent preparation*. 
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King, B. (1989) Educational Development and the DECs, Paper presented at 
Distance Education for Training in Bxisiness and Industry, Australia and South 
Pacific External Studies Association 9th Biennial Forum, 10-14 July, Gippsland 
lAE. (author synopsis) 

•The establishment of Distance Education Centres by the Commonwealth 
Government poses significant problems for the processes of course 
development in a number of tertiary institutions. In particular, the 
relationship between educational author and those with production expertise 
will be altered in the development of external courses. This paper attempts to 
clarify likely areas of difficulty, to suggest a conceptualisation of the process 
of educational development in DEC-specialist provider relationships, and to 
reflect on the practical application of such process for Australian higher 
education institutions'. 

Kwong, Lee Dow. (1991) Academic Standards Panels in Atistralia, paper 
presented at the Hong Kong Council for Academic Accreditation International 
Conference, IS-l? July 1991. 

In this paper the author discusses the contribution made to Australian 
Universities by Academic Standards Panels but points out that other forms of 
evaluation must and do make an important contribution. Such forms of 
evaluation in particular include performance indicators. 

The author concludes that 'provided the panels and academic standards 
program is seen as a major means, but not the only means of maintaining 
consistency, comparability and some interactive dialogue about standards, it is 
likely to serve the Australian system of institutions well in the coming years'. 

Lewis, R. (1989) What is 'quality' in corporate open learning and how do we 
measure it?. Open Learning, 4, 3, 9-1 

This article 'examines notions of quality in open learning for corporate users 
and how this can be evaluated*. The article discusses the need to ensure 
quality in open learning and points out the importance of assessing resultant 
change and benefits and their general relevance to training and evaluation. 
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Linfoot, K. and Todoroski, Naum. (1991) 'Recruitment and training of school 
psychologists for remote or isolated areas'. Distance Education, 12, 2, 286-296. 

Marginson, S. (1990) Pitfalls of Output Measurement, Australian Journal of 
Education 34, 2, 137-144. 

This paper discusses the pitfalls of output measurement. Educational output is 
being judged in economic terms reflecting demands for 'greater accountability, 
tighter administration and identifiable results*. This approach is generally 
associated with budget cut backs and industrial conflict. However, as the 
author points out it does not have to be so'. Marginson states that much of 
the problem with the measurement of educational output stems from the fact 
that teaching productivity has been viewed in terms of student teacher ratios 
rather than in terms of quality of learning. This results in the muddled type of 
thinking whereby* - the higher the student: teacher ratio, the higher the 
productivity*. Outputs must be viewed in terms of objectives. Marginson goes 
on to discuss the limits of measurement within the education system because 
of the complexity and collectiveness of the process. *It is hard to separate out 
the effects of particular inputs or to measure the relationship between part of 
input and part of outputs. In education, as with most production in service 
industries, production and consumption are simultaneous; teaching and learning 
are the same process*. The author concludes the article with a listing of 
factors which need to be addressed if output measurement in education is to be 
more effective. 

Mills, R. and Paul, R. (1991) Putting the Student First: Management for 
Quality in Distance Education, Draft Paper for Cultural Reflections in 
Distance Education 2, (forthcoming). 

This paper discusses the application of Total Quality Management (TQM) to 
Distance Education and in particular points out that *a total quality programme 
must start at the top* It must be more than the latest device to motivate staff 
to provide a better service, but a fundamental value which is both espoused and 
exemplified by the senior management of the institution*. It identifies the 
measures being taken in higher education institutions in the United Kingdom 
and also Canada regarding quality issues. It also describes the application of 
TQM to two distance education institution; The Open University in the United 
Kingdom and Athabasca University. 
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The authors conclude that the issues surrounding the application of TQM must 
focus on the quality of the learning process rather than more identifiable 
products because •••• process is the product in education*. This makes the 
whole process very complex and demanding. 'Ultimately, it takes value driven 
leadership, leadership which is prepared to plan and build for the long run and 
about which there is no question of other priorities when it comes to resource 
allocation and policy decisions*. 

Moses, I. (1989) Is Performance Management Appropriate in a Learning 
Institution? Journal of Tertiary Educational Administration, 11, 2, 127-^141. 
In this article Ingrid Moses provides a definition of Performance Management. 
She then discusses the nature of universities and colleges as learning 
institutions and how this impacts upon Performance Management. In particular 
she points out the need to recognise and assess the contributions of not only 
academic staff but also professional, and support staff within the institution. 
Finally she concludes that Mf we accept quality maintenance as something 
positive or something which is contributing to the very goals of the institution, 
then we have to introduce performance appraisals as loosely structured 
systems so that they can respond to the individual in their various roles within 
one departmental context, one institution*. 

Partington, G. (1990) Quality and Schools Education Research and 
Perspectives, 17, 2, 83-97. (author synopsis) 

'Although education and a fortiori quality in education are contestable 
concepts, every coherent educational theory or practice requires publicly 
available criteria and standards of what it predicates as quality. Concepts of 
quality in four different clusters of educational theories are discussed. It is 
shown that absence of explicit and understood standards of quality is especially 
destructive in the education of groiq>s perceived to be disadvantaged and that 
Australian education is sorely deficient in the conceptualization and provision 
of standards of educational quality*. 

Paul, (1990) Towards a new measure of success: developing independent 
learners. Open Learning, 5, 1, 3J-3S. 

This article is extracted from Ross Paul*s book Towards open management: 
leadership and integrity in open learning and distance education. 'This article 
is derived from a chapter on 'Managing for success: learner interaction and 
independence'. It follows an analysis of such standard measures of success as 
completion and persistence rates and advocates a higher order objective - the 
development of independent learners*. 
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Pollitt, C. (1990) Measuring University Performance: Never Mind the Quality, 
Never Mind the Width, Higher Education Quarterly, 44, 1, 60-^1. (author 
synopsis) 

'After setting performance measurement in the context of the broader changes 
affecting universities the paper enquires how far the actual measures used to 
date by British universities reflect that wider environment. Detailed analysis 
of the indicators concerned indicates that most - particularly those developed 
by the CVCP/UGC - are narrowly conceived The research selectivity 
measures of the UGCAJFC are more ambitious, but even these take little 
account of "value added" issues - and entirely fail to address teaching. 
Comparison with the best of US practice indicates that greater sophistication 
is technically achievable. The failure, as yet, of the British system fully to 
e3q>lore issues of quality and relevance may be connected to the different 
structijre of the higher education sector in Britain, and in particular to the 
combination of a powerful, economy minded central government with weak 
consumer power*. 

Power, C. Participation as an Education Performance Indicator in Wyatt T and 
Ruby A (1988) Indicators in Education: Papers From the First National 
Conference, Australian Conference of Directors-General of Education, Sydney, 
147-162, 

This paper focusses on participation as a performance indicator. It begins by 
examining the concept of participation as a performance indicator, then looks 
at some of the methodological issues involved, and finally at some of the 
political issues which need to be addressed*. The author believes *that 
carefully developed and responsibly used sets of indicators and indicator 
systems have a iOle to play in helping us understand how education systems 
operate, to identify issues and to pii^)oint problems, to enrich the decision 
making process at systems level to provide a more adequate basis for reporting 
on progress towards key systems outcomes*. 

He concludes that society roust clarify is e^q)ectations of schools and while 
actively developing performance indicators recognise both their strengths and 
limitations. 

Ramsden, Paul. (1991) A Performance Indicator of Teaching Quality in Higher 
Education: The Course Experience Questionnaire, Studies in Higher Education, 
16, 2, 129--150. (author synopsis) 

•Performance indicators in higher education have focused chiefly on research 
outputs. They have largely ignored the teaching function of imiversities and 
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colleges. This article outlines the development of a student evaluation 
instrument designed to measure the teaching performance of academic 
organisational units. The theory of teaching and learning that underlies the 
Course Experience Questionnaire (CEQ) is described. The instrument's 
statistical qualities and its ability to discriminate intelligibly between 
different courses are discussed in the context of results from national trials in 
Australia higher education. The principal conclusion reached is that CEQ 
offers a reliable, verifiable and useful means of determining the perceived 
teaching quality of academic units in systems of higher education that are 
based on British models. Several technical and political issues remain 
unresolved in its application as a performance indicator'. 

Ross, B. (1990) Quality and Standards in Distance Education, ASPESA News, 3, 
17, 4^5. 

This article discusses the possible outcomes of the Project to Investigate 
Quality and Standards commissioned by the NDEC. In particular, it points out 
a concern with the production of 'an inventory of procedures and standards 
employing, where appropriate, performance indicators\ The author suggests 
that this may result in a belief and approach to these procedures as if there is 
one correct model of distance education. This may be to the detriment of 
students with varying needs. Finally the author concludes with the hopeful 
comment that the project will focus on the quality of student learning and ... 
produce ... standards relating to procedures for reviewing activities in the light 
of learning outcomes'. 

Scriven, B (1990) Australia Restructures Distance Higher Education, ICDE,2, 
21^23. 

This article presents an overview of the response in distance education since 
the Government's 1988 release of the policy statement on higher education. It 
describes the process by which the eight distance education centres were 
chosen. It then asks what the future of distance education will be in light of 
the governments funding policies and its intention to reduces the number of 
distance education centres. 
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Scriven, Bruce R. Quality in Distance Education, Keynote Address ASPESA 
Forum, Bathurst. (author synopsis) 

'In this paper I have provided a brief summary of some recent statements made 
about quality in higher education and in distance education in particular. 
Quality distance education is seen as being influenced by a complex 
combination of factors which can be broadly categorised under the three 
headings of course materials, course teaching and administration. The 
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determination of what comprises quality distance education is a legitimate 
concern of learners, course writers and presenters, other academics and those 
who place trust in the results. Particular aspects of teaching and 
administration which I believe influence quality of distance education provision 
are discussed*. 

Sherman, R. (1990) The Relationship between Quality Circles and Teacher 
Satisfaction, Educational Research Quarterly, 14, 2, 53-56. (synopsis) 
'This study examined the relatio^iship of teacher participation in Quality 
circles to teacher satisfaction. Sixty four volunteers from seven elementary 
schools in a large suburban school district in Texas were randomly assigned to 
either a control or experimental group. The experimental participants formed 
nine Quality circles of between four and eight members with one member per 
circle being trained as a circle leader. The Job Diagnostic Survey was 
administered twice to the participants, a series of participant observations 
were conducted and structured interviews were held with participants after the 
exf)eriment was concluded. Analysis of covariance was used to test 
relationships among the study variables; the structured interviews yielded 
percentages of responses and a record of verbatim responses. No relationship 
was found between participation in Quality Circles and meeting an individual's 
self esteem needs for achievement, recognition or growth*. 

Sizer, J. (1988) In search of excellence - performance assessment in the United 
Kingdom. Higher Education Quarterly, 42, 2, 152-16L (author synopsis) 
*It is argued that developments in institutional, departmental and individxial 
performance assessment in the United Kingdom cannot be considered in 
isolation from pressures to change from free, oligarchic, consensias models and 
administrative styles of management towards a shared commitment to a 
dominant and coherent entrepreneurial and market economy model culture 
with executive styles of management, which implied institutional and 
managerial performance being assessed in terms of economy, efficiency and 
effectiveness with increased emphasis on effectiveness^ 

Snath, M. (1988) Educational Indicators, Phi Delta Kappan, March, 487-^91. 
This article provides a definition of educational indicators, an overview of 
their use and a discussion of possible future trends. The author identifies 
technical and political problems with the use of indicators and points out the 
difficulties with national comparisons of state institutions. His prediction of 
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the future use of educational indicators include an improvement in data 
gathering techniques and an integration of federal and state efforts to collect 
this data* 

Tate, O. (1986) Monitoring and Evaltiating the performance of a distance 
education institution. Epistolo Didaktika, 2, 77''103. 

•This paper d^.scusses some issues involved in monitoring and evaluating the 
performance of a distance education institution, with particular reference to 
the aims of the institution, course materials and teaching and throughput of 
student work, Although distance education institutions differ from each 
other ... they exhibit many similarities and common elements'. This paper 
identifies and discusses these similarities. 

The paper concludes that *a distance education institution is considered as a 
system with identifiable, interacting subsystems. A data model can be 
developed and used to monitor and evaluate processes and outcomes. This 
involves identifying significant, representative, analysable, accessible and 
objective data and indicators of performance; thus the effectiveness and 
efficiency of sub-systems and the institution can be assessed. Evaluative 
judgements consider performance in terms of educational aims and 
achievement, statutory requirements, societal expectations, the national 
educational context, ability to respond to individual needs, student, parent and 
staff expectations, and the cost of the operation. Monitoring and evaluative 
procedures must be non threatening and seen as necessary. Reforms and 
innovations arising from evaluation are most likely to be successfully 
implemented if staff are involved in evaluation, problem identification, 
formulating solutions and developing procedures*. 

Viljoen, J., Holt, D. and Petzall, S. (1990) Quality Management in an MBA 
Program by Distance Education, paper presented at the ANZAME Conference, 
Launceston, Dec 1990.(znthoT synopsis) 

•This paper uses a qualitative, longitudinal methodology to focus on the 
creation of a consumer (student) oriented framework for quality in the Deakin 
University MBA by distance education. Quality is viewed as the outcome of a 
complex set of interactions between staff, students, learning materials and 
siq)portive educational method. The notion of students acting as passive 
recipients of '•quality'^ teaching is rejected. The convention is made that 
quality is a primary means of gaining competitive differential advantage in a 
specified segment of the MBA market, only if quality is defined by student 
perceptions. The research findings enable the development of a student 
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learning Framework of Quality which demonstrates the need for efficient 
learning, opportunities for student interaction with learning materials, with the 
workplace, with teaching staff, and with study groups and residential schools. 
The framework is analysed in terms of the value chain concept and 
implications are drawn for the successful future management of the Deakin 
MBA program*. 

Vttlarroel, Armando. (1988) Planning of distance-learning projects. Prospects 
18, 1, 54-61. 

In establishing open universities it has been necessary to put forward *a good 
social case' to justify their development. 'The argument most frequently 
advanced has always been that because they are flexible and potentially 
inexpensive to operate, they can reach groups of people who because of their 
geographical location and position in society would otherwise have virtually no 
access to education'. Initiators of these institutions are also preoccupied as to 
how best to put into operation a new teaching concept. 

Bearing in mind the key importance for distance-teaching programmes of 
(these) considerations, we shall in this article conduct a global analysis of their 
influence and of the way in which they plan their activities. We shall begin 
with a summary description of the characteristics which ... should be taken 
particularly into account for planning purposes. We shall then refer to the use 
of systems analysis as a key theory and instrument for the planning of distance 
education programmes and, finally in the light of what has been said, we shall 
identify the subsystems that are most characteristic of these programmes and 
their effect on planning'. 

Vroeijenstijn, A.L (1991) External Quality Assessment: Servant of Two 
Masters? paper presented at the HKCAA International Conference on Quality 
Assurance in Higher Education, Hong Kong, 15-17 July, 1991. (author synopsis) 
'Opinions about the way External Quality Assessment (EQA) in the Netheriands 
is developing, differ. In this paper a description is given as to why people are 
looking at EQA from different angles. After handling this question a 
description is given of the system of EQA at Dutch universities. 

The comments from the side of the Government and the Inspectorate are 
analyzed. The conclusion is that the Government and the Inspectorate are 
expecting too much from the work of external experts and the reports they 
publish. Visiting committees cannot serve two masters. Assurance of quality 
is more important than the exact measurement of it'. 
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West, L.H.T. (1988) Strategic Planning, Performance Indicators, Research 
Policy: the new issues facing university management. Higher Education 
Research and Development, 7, 1, 5-17. (author synopsis) 
•Higher education systems throughout the world are coming under the joint 
influences of continuing reduced resources, demand for research to solve short 
term economic problems, and the perceived value in the application of business 
models of management. These influences have given rise to several related 
movements: strategic planning, performance indicators, research policy. The 
paper provides a discussion of each of these movements'. 

West, LM.T. and Hore, T. (1989) The impact of higher education on adult 
students in Australia: Part 1. Employment. Higher Education, 18, 3, 341-'352. 
•In part 1 of this study, we described the impact of higher education on aspects 
of the employment of a national sample of adult students (aged 25 plus), who 
enrolled in higher education in 1978, The sample and the research design is 
described in part 1* 

In this paper we describe changes in a range of measures relating to the person, 
and make the claim that some part, at least, of these changes can be 
attributed to the impact of higher education. 

Three broad aspects of the potential impact of higher education on adult 
students as persons were investigated. The first asi>ect concerned their 
personal development, including changes in Si'ttitudes and values; the second 
was to do with changes in their abilities, and the third with possible social 
impacts*. Although a longitudinal design would have been preferred for this 
study self report data was collected by asking 'respondents to reflect about 
their response and try to answer as they would have before they began and 
now*. 

Williams, Peter R. (1991) The CVCP Academic Audit Unit, University of 
Birmingham, roneo, 1-17. (author synopsis) 

•The first part of this paper provides a description of the origins, scope and 
method of the Academic Audit Unit (AAU) established the Committee of 
Vice-chancellors and Principals the United Kingdom (CVCP) in 1990, It 
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continues with an account of the Unites first eight months* work, to the 
completion, in May 1991, of a series of five pilot audits, and ends with some 
comments on two different sets of proposals for future quality assurance 
arrangements in British higher education recently published by the Government 
and by the Labour Party'. 

Wright, P.W.G. (1989) Who defines quality in higher education? Reflections on 
the role of professional power in detemdning conceptions of quality in English 
higher education. Higher Education, 18, 2, 149^165. (author synopsis) 
'In recent years British higher education has come under considerable financial 
and political pressure which has led increasingly to the use of various 
indicators for the measurement of quality. Although the use of such measures 
has often been seen as leading to encroachments on academic autonomy, the 
paper argues - drawing in concepts from the sociology of power and the 
professions - that such measures still, essentially, derive from, and reproduce 
imquestioningly, the tacit assumptions and professional self interest of 
academics. This point is illustrated through a brief consideration of the most 
commonly recommended performance indicators and by reference to the work 
of the Council for National Academic Awards in the review and validation of 
courses. Attention is also given to the stimulation of market forces as a means 
of quality control. The paper concludes with a plea for greater pluralism 
within British higher education and argues that several factors make this more 
likely. These include demographic change, the increasing involvement of 
industry and the emergence of diverse sources of funding for British HE 
including the Manpower Services Commission (now the Training Commission)'. 

de Weert, E. (1990) A macro analysis of quality assessment in higher 
education. Higher Education. 19, 1, 57^72. (author sfynopsis) 
'In most West European coimtries a quality control system for higher education 
is in a process of development. This article examines some general 
characteristics of the emerging structures, how quality is defined and what 
impact this definition has on internal evaluation. 

^ n the basis of a more substantive conceptualization of the term quality, the 
current trend of a quality control system in which quality is predominantly 
defined from one central power centre is questioned. Important 
methodological and substantive weaknesses inherent in this centralist model 
will be discussed. 
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In the final section a shift towards a quality control system is advocated in 
which other interests and perspectives are taken into account*. 

Wyatt, T. and Rvby, A. (1989) Education Indicators for Quality, Accountability 
and Better Practice, Papers from the Second National Conference, Australian 
Conference of Directors^-General of Education, Sydney, (author synopsis) 
*This book is based on selected papers given at the National Conference on 
Indicators and Quality in Education... The conference aimed at generating 
further discussion within the educational and wider community about the 
quality of education and ways to monitor, measure and improve this quality. 

The papers selected for publication in this volume are those considered to give 
broad conceptual coverage to these themes, or which have direct relevance to 
the specific issues of educational accountability and quality and the role of 
indicators*. 

Yorke, Mantz. (1991) Performance indicators: towards a synoptic framework. 
Higher Education, 21, 23S'-248. (author synopsis) 

'This paper sets out to sketch a synoptic framework within which performance 
indicators may be used within higher education. 

The evidence to date suggests that the use of performance indicators has not 
adequately taken into account a number of factors - the level within the higher 
education system at which they are being used, their practicability, and the 
inter-relationship between both quantitative and qualitative data. The 
effective use of perform mce indicators is discussed in terms of exercises in 
which judgements have tc be made upon complex sets of data rather than upon 
a narrow range of parameters, with significant implications for what is 
managerially feasible*. 
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